> Is it true that
>
> o in order to access an existing shared library I need to either
>   create a new primitive (named or numbered) or use the FFI
>   framework.
Yes, in practical terms that is true. There _are_ other ways but not
ones I'd advise people to use.
>
> o the VM is not multithreaded, i.e. squeak will be frozen while the
>   call is being executed.
Yes and no. Some platforms use threads for i/o handling but the vm per
se is not multithreaded.
>
> What can I do if I have a library function that does not return
> instantly ? Would it be okay to create a thread in a wrapper C code ?
Yes  _IF_ and only if conditions such as the following apply:-
a) the plugin is for a very specific thing that only works with the
platform, so portatbility is irrelevant. For example it would be
entirely ok to write a VM plugin that interfaced with Windows-specific
APIs and to use Windows specific threading in it. After all, it won't
be any use on Mac or linux or RiscOS.
b) the plugin is private and you have no intention of ever letting
anyone else use it - then you can do anything you please!
c) you are able to wrap the calls in such a way that non-threaded
systems can still use them even if it is a bit inconvenient. See the
examples of sqResolverStartAddrLookup() in the unix SocketPlugin file
'sqUnixSocket.c'.
d) you are able to defend yourself against the machete waving hordes
that can't use your code because they don't have thread-enabled
systems. :-)
tim

Actually you forgot the most important condition:
e) You do not pass any arguments around which may be relocated due to 
GC activity. In my understanding that's the most important reason why 
we don't use any kind of threaded calls within primitives.
Cheers,
  - Andreas

> Is it true that
>
> o in order to access an existing shared library I need to either
>   create a new primitive (named or numbered) or use the FFI
> framework.
No.
You could also use OSProcess and open a pipeline to an external
program that uses the shared library.
> o the VM is not multithreaded, i.e. squeak will be frozen while the
>   call is being executed.
Yes.
> What can I do if I have a library function that does not return
> instantly ? Would it be okay to create a thread in a wrapper C code
Yes.
You should then signal a Semaphore so that the Squeak Process can wake
back up.
--
Ned Konz

Hi Martin--
 I wrote a plugin that uses POSIX threads. I allocate all the stuff 
that the threads care about on the heap, then store the addresses in
ByteArrays that my Smalltalk objects hold onto. A Smalltalk object can
then invoke primitives which use those addresses to manipulate the
threads. Smalltalk processes are synchronized with host threads via
Smalltalk semaphores, which the interpreter proxy can signal.
 Extending the virtual machine to use multiple host threads is no
problem at all.
-C
p.s. The only platform that can't deal with threads is Risc-OS. I look
forward to the day when we are liberated from it (i.e., when Tim stops
using it :) or when the POSIX project there is done.

>Martin Drautzburg

> This concerns only smalltalk objects, right ? Other memory allocated
> by my plugin will not be touched by the GC ?
>
> Still I wonder ... If the system is frozen as long as the plugin
> executes the GC has no chance to run. I can see how threads will be a
> problem, but don't the same concerns apply to any stateful code ?
Technically the VM, allocates/malloc/etc one huge memory segment to
contain the contents
of your.image plus head room for youngspace.
Most unix vm (linux/bsd/os-x) mmap 1GB (or 512mb (mac)) and write the
image there, with the
1GB-sizeof(your.image) as youngspace.
The GC then manages memory within that one huge memory segment. The VM
 then of course can
malloc/allocate/etc memory for other things in what is known as the C
heap.
Since the VM is not threaded then within a primitive call, a GC won't
happen, unless you invoke a VM call that results in allocating
smalltalk objects which could then trigger a IGC or fullGC.  There is
a  call, push/popRemappableOops that you can make to record oops
address  that will be remapped on a GC. Usually these are made when
the  primitive is allocating a  number of objects to contain data
coming up  from a primitive call, look at the plugins for an example.
Now for example if in your primitive call, you malloc memory, then
copy  the oops data into those storage areas, then place some
token/indicator  on a thread safe queue, which your other pthread
process is waiting on,  or by passing that mallocated memory to a
pthread somehow you will be  fine.
Getting the data back will require some thought.
Although signalling a semaphore is kinda thread safe, care was taken
by  me a few years back to make it somewhat thread safe to support
Apple's  open transport tcp/ip callbacks but it's not perfect.
Please review the code in Interpreter>>signalExternalSemaphores &
Interpreter>>signalSemaphoreWithIndex:  and see if it passes your
thread safe thresholds. CPUS are WAY faster today that when it was
written, then also consider the issue if your signalSemaphoreWithIndex
 is ignored/forgotten/missed by the VM, because this code doesn't work

just right. Nothing of course prevents you from cleaning this code up
with pthread safe code in your own custom VM.
Making interpreter callbacks in the pthread is fraught with danger,
certainly none that references Oops can be made, unless of course you
build a custom VM that allows that, yes one could pthread safe or spin
 lock the allocation routine then allow allow threads to allocate oops

memory.
Also note that past experience has shown that pthread creation is
expensive, so having a pool of threads to use, or some sort of queue
to  feed a pthread is more efficient that attempting to create dozens
of  pthreads per second.

John M. McIntosh

> This concerns only smalltalk objects, right ? Other memory allocated
> by my plugin will not be touched by the GC ?
Yes.
> Still I wonder ... If the system is frozen as long as the plugin
> executes the GC has no chance to run. I can see how threads will be a
> problem, but don't the same concerns apply to any stateful code ?
Yes it would. However, most primitives/ffi calls require the state 
only for life-time of the primitive call itself in which case this 
isn't a problem. If you do need state to be persistent across 
primitive calls you will have to allocate it on the C heap and pass a 
handle to it back to the Smalltalk side.
Cheers,
  - Andreas