(Rothgar) Re:EverqQuest2.exe 1.8gigs of memory usage?!?!?

[Official Forums]

I'm very excited to hear that many of you are seeing a big improvement to memory usage after the Nov 2nd hotfix. You don't know how big of a weight off my chest that is!

It really was a team effort to make some of these optimizations and we still have more work to do. As I mentioned in another thread, we have a string hashmap system coming soon that will cut down string allocations by half.

I recognize that some of you are still having problems, especially in overloaded guildhalls. We will continue to look for improvements in this area going forward. We really want to put our best foot forward by February when the expansion comes out and our marketing department begins their big acquisition campaign.

Please continue to post your feedback on memory usage in this thread. We're keeping up with it every day even if we don't always get a chance to respond.





LINK

[Illuminator]

Originally Posted by Official Forums

As I mentioned in another thread, we have a string hashmap system coming soon that will cut down string allocations by half.

Aditu should also be doing this if she isn't already.

[the_mo]

why? memory usage of ACT is really a non-issue

[Illuminator]

so is the memory usage of EQ2

[the_mo]

that was a good one :D

[EQAditu]

Well, I'm probably not.

I don't even innately know what Rothgar is getting at, but by the context... something like storing one copy of a unique string and giving data objects an index to the string data instead of storing the actual string? IE, instead of a mob name being repeated 5,000 times consuming say 10 bytes each, you use an integer using 4 or less bytes and have only a bit of overhead for storing the so called collection of strings.

It wouldn't even be really hard to do since I mask everything with properties. Something interesting to look at when I have free time, I suppose.

[Illuminator]

public Dictionary<string,string> SharedHashTable = new Dictionary<string,string>();

public string GetSharedReference(string InputString)
{
    if (!SharedHashTable.ContainsKey(InputString))
        SharedHashTable.Add(InputString,InputString);

    return SharedHashTable[InputString];
}

string InputPlayerName = "Aleraku";
string SharedPlayerName = GetSharedReference(InputPlayerName);
// If the reference already exists in the dictionary, InputPlayerName's version then gets discarded by the GC.

If you're storing the data the way I would, you have potentially millions of duplicate string copies depending on the size of the parsed log file.

[SacDaddy]

Originally Posted by Official Forums

I'm very excited for hash




LINK

That shits the chronic!

[EQAditu]

Originally Posted by Illuminator

public Dictionary SharedHashTable = new Dictionary();

public string GetSharedReference(string InputString)
{
    if (!SharedHashTable.ContainsKey(InputString))
        SharedHashTable.Add(InputString,InputString);

    return SharedHashTable[InputString];
}

string InputPlayerName = "Aleraku";
string SharedPlayerName = GetSharedReference(InputPlayerName);
// If the reference already exists in the dictionary, InputPlayerName's version then gets discarded by the GC.

If you're storing the data the way I would, you have potentially millions of duplicate string copies depending on the size of the parsed log file.

Hmm, I guess my theory was similar but different. Instead of giving an object reference to my data objects, I was suggesting giving an Int16/32. Object references aren't zero-size after all.

I made a little arbitrary test where a program made 10M total strings with 10,000 unique ones. Simply storing the strings consumed ~7.64s and 871M. Using your method consumed ~3.68s and 168M. My supposed method consumed 2.79s and 94M.

Keep in mind I used Int16 and circumstances may have forced me to use Int32 if I had say 70k unique strings. Also my data classes would have had to call the String/Int16 dictionary to resolve the string. So the middle ground is much simpler to implement.

[Illuminator]

My method was quick and dirty but still offered substantial improvement over nothing at all. Its main flaw is that it has to generate a hash for the string every time it does a dictionary lookup, unlike an integer handle. And yes, I would do the Int32 over the Int16. 65K is not an impossible number to exceed.