?

Log in

No account? Create an account
collecting garbage - The Mad Schemes of Dr. Tectonic [entries|archive|friends|userinfo]
Beemer

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

collecting garbage [Jan. 31st, 2006|05:22 pm]
Beemer
I'm making this post mostly so I can cut-and-paste it to a mailing-list, but what the heck, maybe somebody reading this will know the answer, too.

My game has a crash-bug. In certain circumstances (not easily reproducible, ha-ha aren't THOSE the best kinds of bugs), it will use up enough memory to exceed the default JVM heap size, and the whole thing dies with a java.lang.OutOfMemory exception.

Now, this can be pretty easily prevented by running it from the command-line with the flag "-Xmx 256M", which tells the JVM to use more memory. But the problem with that solution is that our intended distribution method for the game is for the users to download it by right-clicking on the link, and run it by double-clicking the icon. Any startup procedure more complex than that is Just Too Hard. Open up a command prompt and type things that have to be spelled correctly? No way.

I guess that means I need to get it to use less memory. Okay, run a profiler on it. Now, our game has some pretty big data objects that we send back and forth between client and server by serializing them to XML. I already knew that the serialization code (which is third-party, and which I'm not going to touch) is pretty slow. Now I'm pretty sure that it's also a memory hog. In particular, I discovered that I can hit the "garbage collect NOW" button in the profiler and the memory usage drops by HALF. And the objects that get collected appear to all be spawned by things in the serialization code.

So... I'm thinkin' that means that I want garbage-collection to run more aggressively.

If I put a "System.gc()" call in at the end of the procedure that serializes the entire gamestate to XML, that should help, right?

Any pitfalls to prompting the garbage-collector to run in the spots where I know a lot of dead objects are being created?

Or am I barking up entirely the wrong tree?

...Bueller? ...Bueller?
LinkReply

Comments:
[User Picture]From: jofish22
2006-01-31 05:18 pm (UTC)
Well *I* like the fact that you can move the word "java" in the sentence "My game runs java out of memory" just about anywhere in the sentence with occasionally a wee bit of punctionation and it makes no difference to the sentence!

java: my game runs out of memory
my java game runs out of memory
my game: java runs out of memory
my game runs java out of memory
my game runs out java of memory [ok this one fails]
my game runs out of java memory
my game runs out of memory - java

kinda like the ploughman homeward wends his weary way.
ok, i guess that didn't help at all. but it amused me.
(Reply) (Thread)
[User Picture]From: melted_snowball
2006-01-31 08:24 pm (UTC)
That's sort of the opposite of "only"; I can't remember the sentence right now, but there are a few for which you can make seven different meanings by placing "only" into a six-word sentence.
(Reply) (Parent) (Thread)
[User Picture]From: jofish22
2006-02-01 09:16 am (UTC)
When I'm directing or teaching theater, I often use the sentence

I didn't steal grandmother's false teeth.

which, by putting emphasis on successive syllables -- without even adding a word -- you can make sentences with six different meanings.
(Reply) (Parent) (Thread)
[User Picture]From: ng_nighthawk
2006-01-31 08:26 pm (UTC)

As a tester. . .

If a programmer told me he had made this change, here's what I would check.

I would check and make sure that universal variables are maintained--unless all the data necessary for the game is serialized. But there's gotta be some server config data that doesn't change from game-to-game. . . I would think. So I would make sure any unserialized data isn't thrown out with the trash. :) Then I would check the serialized output and ensure that all the data integrity was maintained. I would run at least 5 cycles of serialization with different parameters to ensure nothing odd happens.

Reading up on how this gc funx works, it is both cool and incredibly frightening how this memory management system works. But then you're assuming that the programmers of this serialization code were good about releasing something when it should be released, and that they didn't just leave it on the heap assuming it would be available later. If your code passes those two tests, my belief would be that it would be a good fix.
(Reply) (Thread)
[User Picture]From: dr_tectonic
2006-02-01 08:56 am (UTC)

Re: As a tester. . .

You're on the wrong track with the universal/serialized variable stuff. There's no state preserved between games, and everything's getting piped through properly -- that's easy to tell. (And I've got kerjillions of successful serialization attempts.)

I'm pretty certain that it's just that the serialization code is kinda crappy in terms of how it handles memory. Unfortunately, it's not something I can muck around with; I need to fix it from the outside.
(Reply) (Parent) (Thread)
[User Picture]From: goobermunch
2006-01-31 08:48 pm (UTC)
I'd just set up my installer to install the game with a shortcut that defaults with a "-Xmx 256M" flag. In the EULA, in very small type, I'd tell the customer that they're agreeing to let me muck about with how Java works on their system and include language stating that they agree to hold harmless and forever indemnify me for any damage that this may cause to their systems.

But I'm not a programmer, I'm a lawyer.

--G
(Reply) (Thread)
[User Picture]From: navrins
2006-01-31 09:56 pm (UTC)
That's what I was thinking. Not hard in Windows or Unix, and I don't see a downside.

I think System.gc() is also safe, but not always reliable - it is a *suggestion* that "hey, you could run the garbage collector now if you want," but it doesn't *force* garbage collection. I have been bitten by this.

I'm almost certain ng_nighthawk's concerns are unfounded, unless I'm misreading what he's saying. gc() won't collect memory that still has pointers to it.
(Reply) (Parent) (Thread)
[User Picture]From: dr_tectonic
2006-02-01 09:04 am (UTC)
We can't have an installer because 90% of our target market will be playing the game on machines where they don't have permission to install anything.

However, we could probably get away with a self-executing batch file or something like that. Any suggestions on places to find patterns to follow?
(Reply) (Parent) (Thread)
[User Picture]From: dr_tectonic
2006-02-01 08:58 am (UTC)
Ah, but there is no installer.

The primary reason for this strategy is that the default operating environment is a college computer lab where the users don't have permission to install anything -- but they do have permission to save a file to the desktop and run it.

I have an email to the lab's law office asking what kind of disclaimer/license we need for this. Haven't heard anything back yet...
(Reply) (Parent) (Thread)
[User Picture]From: finagler
2006-01-31 11:29 pm (UTC)
Since you get to cut-and-paste to a mailing list, the least I can do is cut-and-paste my mailing-list response back into LJ ;).

I suppose it's possible your JVM would die with an OutOfMemory without first trying to garbage-collect all that it could, but my guess is that you've got an honest-to-god loop somewhere that's actively pointing to all that memory. If so, GCing won't help
fix your bug.

As for launching by double-clicking, Java really does fulfill the promise of "write once, break everywhere." Do you expect all your users to have the JVM installed already? The right version of it? With all the classes you need?

Here's an article that might help:
http://www.excelsior-usa.com/articles/java-to-exe.html

If you don't mind doing machine-specific downloads, you might also try a batch file or launcher.
(Reply) (Thread)
[User Picture]From: dr_tectonic
2006-02-01 09:15 am (UTC)
So far, the answer seems to be that yes, actually, nearly all of our users have a compatible JVM already installed, and those that don't (typically mac and linux users) are sophisticated enough to install it themselves.

But that article is really useful! Thanks!

Suggestions on how to do a simple batch file / launcher? I'm not at all opposed to machine-specific downloads. They're so common that everyone knows how to deal, and it shouldn't be a big deal on our end to generate them...
(Reply) (Parent) (Thread)
[User Picture]From: finagler
2006-02-01 11:21 am (UTC)
I haven't really used Windows since, well since it was called DOS, but can't you just put the java command you want in a file called "double-click-on-me.bat" and then put it in a ZIP file along with the JAR file?

I think the launcher approach lets you make it just a single file rather than JAR + batch file, but I've never done that in Windows-land.
(Reply) (Parent) (Thread)
[User Picture]From: dr_tectonic
2006-02-01 12:02 pm (UTC)
Yeah, but there's enough setup steps already that I really want to make it invisible if I possibly can. "Download this file, then unzip it, then double-click this other file" is guaranteed to have more people get lost than "download this file and double-click it", sadly.

The article you linked to had some suggestions about launchers; I'm going to look into those.
(Reply) (Parent) (Thread)
[User Picture]From: backrubbear
2006-02-03 11:48 am (UTC)
I don't really know squat about java.

That said, does your parser get instantiated once and used many times throughout the program? I.e. something like:

myparser = new XMLparserThing();
myparser....

If so, you might want to delete the parser object in between uses. A lot of parsers are very sloppy about leaving intermediate data structures around to accomodate a number of access methods.

Then again, I may be speaking nonsense. :-)
(Reply) (Thread)