in a Workspace I did
a:= Array new: 100000000 withAll: 1.0
Which gave me an 80% cpu usage.
To verify I browsed a MessageTally and found that incremental GC used
88%. One Smalltalk garbageCollect brought things back to normal.
As far as I remember my swiki readings this is because everything
surviving a GC is moved to old space which is ignored by incremental
GC.
Another interesting observation:
Doing the allocation with immediate GC is quite fast while doing
a:= nil with immediate GC takes *much* longer.
BTW, if you don't have the latest CPU 10 million objects will do.
Win 3.10.6 VM, 3.8.2, 3.9 final and latest 3.10 images.
Do I have to learn from this, that when creating lots of objects I
intend to keep for a while, it's good to force a GC instead of waiting
that time will move my objects to old space?
---
What you need to learn from this is that when you have non-trivial
workloads you need to adjust the GC parameters to fit your application
needs. For example, we use the following settings for Qwaq Forums:
initializeMemorySettings
"Initialize the memory and GC settings to be more in line with QF requirements"
"The following settings affect the rate incremental GCs and tenuring"
"Limit incremental GC activity to run every 40k allocations"
Smalltalk vmParameterAt: 5 put: 40000. "allocations between GCs (default: 4000)"
"Limit tenuring threshold to only tenure w/ > 10k survivors"
Smalltalk vmParameterAt: 6 put: 10000. "tenuring threshold (default: 2000)"
"These settings affect overall memory usage"
"Only give memory back to the OS when we have more than 16MB free"
Smalltalk vmParameterAt: 24 put: 16*1024*1024. "default 8MB"
"Try to keep 8MB headroom at all times"
Smalltalk vmParameterAt: 25 put: 8*1024*1024. "default 4MB"
"These settings describe what to do when we're close to running out of free space"
"Tell the VM that we'd rather grow than spinning in tight GC loops"
Smalltalk gcBiasToGrow: true. "default: false"
"Tell the VM to do a fullGC for good measure if the above growth exceeded 16MB"
Smalltalk gcBiasToGrowLimit: 16*1024*1024. "default: 0"
Cheers,
- Andreas
--
Jon Hylands wrote:
> I can't find any Squeak images that implement #gcBiasToGrow:... Is that
> something you wrote, or part of some package I don't have?
It's been part of the VM for a long time but not exposed to the image in
any shape or form. We added it to Croquet:
SystemDictionary>>gcBiasToGrow: aBool
"Tell the VM to grow after tenures instead of running in a tight loop
where it does IGCs over and over. For some weird reason the primitive
takes an integer not a bool but oh well..."
self primitiveGCBiasToGrow: (aBool ifTrue:[1] ifFalse:[0])
SystemDictionary>>primitiveGCBiasToGrow: arg
"Tell the VM to grow after tenures instead of running in a tight loop
where it does IGCs over and over. For some weird reason the primitive
takes an integer not a bool but oh well..."
<primitive: 'primitiveSetGCBiasToGrow'>
^self primitiveFailed
Ditto for any others that may be missing.
Cheers,
- Andreas
--
Klaus D. Witzel wrote:
> I suggest to activate the #initializeMemorySettings which Andreas posted
> in the squeak-dev and -web .images. TIA.
I wouldn't. These settings were chosen to balance the tradeoffs between
interactive rates (real-time voice and sound processing) as well as
large memory allocations (big chunks of memory for textures etc). For a
web-image (which has no real-time constraints) I would almost certainly
choose different values. And for an image that usually stays below 100MB
in size, I would choose even different values.
For example, for the web image try:
- allocation between GCs: 100,000
- tenuring threshold: 25,000 (maybe even 35,000)
The above will GC less often but take slightly longer (20ms maybe
depending on your speed). But it will tenure much less often and
consequently grow more slowly. This can often compensate for working
sets that are accessed over longer periods of time (I would expect
Seaside to show such behavior).
Cheers,
- Andreas
--
Further to this in Sophie we have a SophieMemoryPolicy which does
passive checking for certain things and
perhaps changes things on the fly
The real issue is when the root table points to many arrays and we
have to scan each Array looking for
the young object, that leads to excessive work in the IGC.
But I know that cases arises when we see (statMarkCount ) >
(statAllocationCount*2) so we force a tenure which fixes the problem.
I experimented with setting goals for the new space object allocation,
and tenuring threshold. This works
ok if you have a particular ms wait for a IGC in mind and perhaps a
server type of load, if you have a server
test suite that mimics your workload then fiddling with Smalltalk
vmParameterAt: 5/6 on the fly might alter
things (good/bad).
calculateGoals
| target |
vmNotSupported = true ifTrue: [^self].
(statMarkCount ) > (statAllocationCount*2)
ifTrue: [[Smalltalk forceTenure] on: Error do: [:ex | ]]. "Tenure
if we think too much root table marking is going on"
true ifTrue: [^self].
" Logic below is not used, still there in case one wants to try it "
"Unused logic to set allocation and tenure limits to reach some sort
of icremental GC ms time"
(statIGCDeltaTime = 0) ifTrue:
[target _ (Smalltalk vmParameterAt: 5)+21.
Smalltalk vmParameterAt: 5 put: target. "do an incremental GC after
this many allocations"
Smalltalk vmParameterAt: 6 put: target*3//4. "tenure when more than
this many objects survive the GC"].
(statIGCDeltaTime > 0) ifTrue:
[target _ ((Smalltalk vmParameterAt: 5)-27) max: 2000.
Smalltalk vmParameterAt: 5 put: target. "do an incremental GC after
this many allocations"
Smalltalk vmParameterAt: 6 put: target*3//4. "tenure when more than
this many objects survive the GC"].
(statIGCDeltaTime < 1) ifTrue:
[target _ (Smalltalk vmParameterAt: 5)+21.
Smalltalk vmParameterAt: 5 put: target. "do an incremental GC after
this many allocations"
Smalltalk vmParameterAt: 6 put: target*3//4. "tenure when more than
this many objects survive the GC"].
(statIGCDeltaTime > 1) ifTrue:
[target _ ((Smalltalk vmParameterAt: 5)-27) max: 4000.
Smalltalk vmParameterAt: 5 put: target. "do an incremental GC after
this many allocations"
Smalltalk vmParameterAt: 6 put: target*3//4. "tenure when more than
this many objects survive the GC"].
John M. McIntosh <johnmci at smalltalkconsulting.com>
Squeak Smalltalk : System : 
Garbage Collector Parameter Adjustment