Squeak SmalltalkJoker Squeak Smalltalk : Image VM OS Application : prevnext Memory Allocation Mac UNIX

If I have image of size, say 20MB, is it loaded into the memory as one 
big chunk? Or is memory allocated on demand somehow? This concerns me 
because I have virtual servers with only 24MB RAM and they run Linux 
Debian with Apache with PHP and still have free memory.

--

The os-x carbon VM calls mmap( NULL, gMaxHeapSize, PROT_READ | 
PROT_WRITE, MAP_ANON | MAP_SHARED,-1,0);

where gMaxHeapSize is 512MB.

Then a small chunk of memory (image size plus slack plus freehead 
room) is allocated out of this by some logic that ensures a full GC is 
done before more memory is grabbed out of this reserved space. Once 
free space goes over a certain limit, the total used memory is 
decreased, this is set from the image. The grow/shrink values and 
logic.  If you use Apple's activity monitor you'll see that Squeak has 
735MB of virtual memory, but only using 27MB of Real Memory.

For os9  9.2.x we invoke OpenMappedScratchFile & MapFileView for 128MB 
(unless the application memory size is > 128MB) then fallback to 
Newptr if the operating system (OS7.5.5-OS9.0.0) does not support 
MapFileView.

For unix if mmap is supported then it does #define MAP_PROT  
(PROT_READ | PROT_WRITE) #define MAP_FLAGS  (MAP_ANON | MAP_PRIVATE) 
mmap(0, heapLimit, MAP_PROT, MAP_FLAGS, devZero, 0) and I believe it 
targets 1GB.

or a malloc() if mmap is not supported.

Again all the rules for grow/shrink apply, as mentioned the values can 
be changed in the image Smalltalk vmParameterAt: 25 put: 4*1024*1024. 
"grow headroom" Smalltalk vmParameterAt: 24 put: 8*1024*1024.  "shrink 
threshold"

which dictates 4MB mostly free, when we run out of memory grow in 
increments to the max size. When we exceed 8MB free , shrink to about 
4MB free.

Which implies for your 20MB image it would allocate about 24MB of real.

The maximum amount of virtual memory can be changed in the carbon os-x 
version and via command line in the unix version to limit the virtual 
memory allocated (perhaps depends on OS) to back the usage.

Technically you could startup the linux VM and say it has a 32MB max, 
now the issue (maybe) is that I'd suspect you need 30ish MB of virtual 
memory  to run a 20MB image. Because of (rare) fullGC work this 
requires touching all 20MB, and depending on the random usage of 
objects in your image again perhaps all 20MB is need, plus of course 
the 4MB being used for New Space allocation, plus space for the 
virtual machine execution logic.  Shrinking your image down to 8MB 
would be a better thing. Really you need to see what happens and 
observe what it's doing to page in/out activity.

In general having a smalltalk image do paging is a bad thing.

which dictates 4MB mostly free, when we run out of memory grow in 
increments to the max size. When we exceed 8MB free , shrink to about 
4MB free.

Which implies for your 20MB image it would allocate about 24MB of real.

The maximum amount of virtual memory can be changed in the carbon os-x 
version and via command line in the unix version to limit the virtual 
memory allocated (perhaps depends on OS) to back the usage.

Technically you could startup the linux VM and say it has a 32MB max, 
now the issue (maybe) is that I'd suspect you need 30ish MB of virtual 
memory  to run a 20MB image. Because of (rare) fullGC work this 
requires touching all 20MB, and depending on the random usage of 
objects in your image again perhaps all 20MB is need, plus of course 
the 4MB being used for New Space allocation, plus space for the 
virtual machine execution logic.  Shrinking your image down to 8MB 
would be a better thing. Really you need to see what happens and 
observe what it's doing to page in/out activity.

In general having a smalltalk image do paging is a bad thing.

--

The size of Squeak's heap will initially be the size of the image plus 
a little operating room past that.  After that, it will additionally 
use whatever memory your server allocates.  The used memory is in one 
sequential chunk of *virtual* memory, but being virtual memory, this 
can be located anywhere in RAM or in swap.

Squeak won't demand page out of its image file.  Instead, when you 
load an image to begin with, Squeak goes through the entire heap a few 
times to fix up some various things.  Thus, starting a large Squeak 
image (relative to available RAM) will probably cause a little 
thrashing. After that, a lot of the memory will likely drift out of 
RAM and into swap.

If you avoid full garbage collects, then Squeak's virtual memory will 
act a lot like a C program's.  The used spots will stay in RAM, and 
the unusued spots will quietly page out to disk.

If you do end up with a full garbage collect, Squeak will page through 
the whole heap again, causing thrashing just like when you first start 
the package only a little worse (worse, because the traversal will be 
in the order of the object pointers, not necessarily sequentially 
through the heap).

Most servers are unlikely to do a lot of full GC's, though there is 
always the risk.  If you are running into full GC's, though, you could 
play with the VM parameters on memory allocation.