GNOME Bugzilla – Bug 562575
Set correct GC parameters and max memory usage
Last modified: 2018-01-27 11:42:58 UTC
The trick to this bug is to define "correct." Our current code is: JS_NewRuntime(1024*1024 /* max bytes */); This leads to a lot of catastrophic out-of-memory errors. (Our app may be leaking, but it's not hard to imagine hitting this limit without leaking.) If I'm looking in the right place, mozilla does this: const uint32 gGCSize = 32L * 1024L * 1024L; /* pref? */ mRuntime = JS_NewRuntime(gGCSize); if(!mRuntime) return NS_ERROR_OUT_OF_MEMORY; // Unconstrain the runtime's threshold on nominal heap size, to avoid // triggering GC too often if operating continuously near an arbitrary // finite threshold (0xffffffff is infinity for uint32 parameters). // This leaves the maximum-JS_malloc-bytes threshold still in effect // to cause period, and we hope hygienic, last-ditch GCs from within // the GC's allocator. JS_SetGCParameter(mRuntime, JSGC_MAX_BYTES, 0xffffffff); I can't say I really understand the comment in the mozilla code. There are two GC parameters: JSGC_MAX_BYTES and JSGC_MAX_MALLOC_BYTES. The arg to JS_NewRuntime() sets both of them at once. There are no docs I can find except the comments in jsapi.h: typedef enum JSGCParamKey { /* Maximum nominal heap before last ditch GC. */ JSGC_MAX_BYTES = 0, /* Number of JS_malloc bytes before last ditch GC. */ JSGC_MAX_MALLOC_BYTES = 1, /* Hoard stackPools for this long, in ms, default is 30 seconds. */ JSGC_STACKPOOL_LIFESPAN = 2 } JSGCParamKey; But for me this just raises questions like "what does 'nominal' mean here?" and "when is JS_malloc used vs. something else?" Another factor, I would think triggering GC at some number of bytes SpiderMonkey knows about is almost nonsensical, because in a gjs environment each JS object is pointing to a potentially much larger C object ... perhaps we want to manually trigger GC based on something other than how much memory SpiderMonkey thinks is allocated. Though I'm not sure what it would be based on. It would seem xpcom has this issue also so perhaps there's a solution, or perhaps it just doesn't matter.
I found the following comment inside Spidermonkey: http://mxr.mozilla.org/mozilla-central/source/js/src/jsapi.cpp#2507 /* * We run the GC if we used all available free GC cells and had to * allocate extra 1/3 of GC arenas since the last run of GC, or if * we have malloc'd more bytes through JS_malloc than we were told * to allocate by JS_NewRuntime. * * The reason for * bytes > 4/3 lastBytes * condition is the following. Bug 312238 changed bytes and lastBytes * to mean the total amount of memory that the GC uses now and right * after the last GC. * * Before the bug the variables meant the size of allocated GC things * now and right after the last GC. That size did not include the * memory taken by free GC cells and the condition was * bytes > 3/2 lastBytes. * That is, we run the GC if we have half again as many bytes of * GC-things as the last time we GC'd. To be compatible we need to * express that condition through the new meaning of bytes and * lastBytes. * * We write the original condition as * B*(1-F) > 3/2 Bl*(1-Fl) * where B is the total memory size allocated by GC and F is the free * cell density currently and Sl and Fl are the size and the density * right after GC. The density by definition is memory taken by free * cells divided by total amount of memory. In other words, B and Bl * are bytes and lastBytes with the new meaning and B*(1-F) and * Bl*(1-Fl) are bytes and lastBytes with the original meaning. * * Our task is to exclude F and Fl from the last statement. According * to the stats from bug 331966 comment 23, Fl is about 10-25% for a * typical run of the browser. It means that the original condition * implied that we did not run GC unless we exhausted the pool of * free cells. Indeed if we still have free cells, then B == Bl since * we did not yet allocated any new arenas and the condition means * 1 - F > 3/2 (1-Fl) or 3/2Fl > 1/2 + F * That implies 3/2 Fl > 1/2 or Fl > 1/3. That can not be fulfilled * for the state described by the stats. So we can write the original * condition as: * F == 0 && B > 3/2 Bl(1-Fl) * Again using the stats we see that Fl is about 11% when the browser * starts up and when we are far from hitting rt->gcMaxBytes. With * this F we have * F == 0 && B > 3/2 Bl(1-0.11) * or approximately F == 0 && B > 4/3 Bl. */ Quick summary: - The JSGC_MAX_BYTES limit is only hit if the current GC run is taking more than 4/3 of the last run. This seems to be a behavior optimized for firefox. - The JSGC_MAX_MALLOC_BYTES limit is absolute.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gjs/issues/43.