Squeak SmalltalkJoker Squeak Smalltalk : Persistency : prevnext Databases Persistency

Databases and persistence

This is a place to talk about and reference persistence-related code and 
applications in Squeak.

----------------------------------------------------------------------------

Squeak image

Squeak (as all Smalltalk implementations worthy of that name) implements 
persistency in a way which is (almost) unseen in any other computational 
environment. It does not need databases, XML or any similar superfluous 
stuff to be persistent. Just try:

Smalltalk snapshot: true andQuit: false.

You can also create a proces which saves the image periodically instead of 
you.

[[true] whileTrue:

        [(Delay forSeconds: 1000) wait.

         Smalltalk snapshot: true andQuit: false

        ] ] forkAt: 49.

----------------------------------------------------------------------------

File Dictionary

The File Dictionary is an extremely simple database structure. It allows a 
large (larger than memory) file to be treated as a Dictionary by keeping an 
in-memory index. It uses native Squeak serialization (DataStream and 
ReferenceStream version available). It does not support 
transactions/concurrency/etc. For details see FileDictionary

----------------------------------------------------------------------------

ImageSegments

ImageSegments may also be used as a simple kind to store objects while at 
the same time keeping the footprint of your main image low. Details see : 
Introduction to ImageSegments.

----------------------------------------------------------------------------

OmniBase

OmniBase is a production-strength object database. Cees de Groot is porting 
it to Squeak, details can be found on the OmniBase Swiki.

----------------------------------------------------------------------------

MinneStore

MinneStore is a free object database that has been ported to Squeak. 
Information about MinneStore can be found at: 
http://minnestore.sourceforge.net/ The latest files are in 
http://sourceforge.net/projects/minnestore/

Now it works in 3.2.

Minnestore has been around for along time and seen a fair bit of use. It 
feels somewhat like using a relational DB - you have to declare type and 
relationship info for your model. In return, you get automatic indexing and 
query facilities. It supports transactions.

Questions:

What is the state of concurrent access?. Not finished, still can be two 
users locked tryng to change the same object. Does it has GC? Not by now 
(02/25/2003)

----------------------------------------------------------------------------

GemSqueak

Squeak Client for GemStone/S. GemStone is an industrial-strength (and not 
inexpensive) object-oriented database based on Smalltalk. Please visit the 
GemSqueak page to learn more about it.

----------------------------------------------------------------------------

Kats

Kats is a transaction service for smalltalk that is used in conjunction 
with persistency and ORB services. It is used to help manage the 
synchronization of object state with external systems. See Kats for more 
details.

----------------------------------------------------------------------------

MySQL

Bolot Kerimbaev is maintaining the Squeak port of Josh Miller's VisualWorks 
MySQL driver. See the MySQL page for more details.

----------------------------------------------------------------------------

PostgreSQL

A "PostgreSQL Client for Squeak" can be found at 
http://members.rogers.com/yanni/pgsqueak I'm finding the name unwieldy, so 
future versions will call it "pgsqueak".

Latest release 0.9 is available. The same code runs on VAST. -- Yanni Jew 
(Apr 26, 2003).

I updated the links above (a newer 0.8.0 release is available). -- Yanni 
Jew (Dec 14, 2002).

How about "pgsqueakl"? --Allan Baruz

It took me a minute to get it. Maybe "pgSQueakL" would be more obvious. 
--Yanni Jew (Apr 14, 2002)

I've just released a Squeak PostgreSQL interface. See 
http://www.io.com/~jimm/downloads/squeak_postgresql/index.html. Please let 
me know about any problems (jimm_at_io.com). --Jim Menard (Sep 13, 2002)

----------------------------------------------------------------------------

GOODS

Avi Bryant wrote the beginnings of a Squeak interface to the GOODS object 
database.

----------------------------------------------------------------------------

ODBC

ODBC for Squeak: Very simple framework for ODBC support in Squeak

----------------------------------------------------------------------------

SQLite

Maybe someone is able to wrap SQLite a free database server (twice as fast 
as Postgres), availabe as a C library, just 3 simple api calls (connect, 
disconnect, execute), full SQL 97 see http://www.hwaci.com/sw/sqlite/

There's an ODBC driver for SQLite that works with the ODBC API for Squeak, 
above

There's also now a simple FFI wrapper for it at 
http://beta4.com/squeak/aubergines/source/SQLite.st

----------------------------------------------------------------------------

SIXX - Smalltalk Instance eXchange in XML

Smalltalk object XML serializer/deserializer SIXX 
http://www.mars.dti.ne.jp/~umejava/smalltalk/sixx/index.html By using SIXX, 
you can store and load Smalltalk objects in a portable XML format. It is 
like #storeString/#readFrom:, but it resolves shared or cyclic references 
and never creates copies. Runs on Squeak, VisualWorks 5&7, Dolphin 
Smalltalk XP. (SqueakMap-entry)

----------------------------------------------------------------------------

BerkeleyDB Plugin

Actually, this is more than just a plugin...it's an object database 
implementation using BerkeleyDB. See: http://spair.swiki.net/27

----------------------------------------------------------------------------

Magma

Magma is a multi-user object database designed for Squeak systems from the 
ground up. It provides transaction support and transparent access.

See Magma. Magma FAQs

----------------------------------------------------------------------------

Glorp

Nevin Pratt ported Glorp to Squeak. His porting "diary" can be found at: 
http://home.earthlink.net/~nevinpratt/GlorpPort.doc

The Squeak changeset can be found at: 
http://home.earthlink.net/~nevinpratt/glorp.cs

Don't use the above. Instead use my newer port at: 
http://www.smalltalkpro.com/squeakextras.html Or, you can get it from 
SqueakMap.

Nevin

----------------------------------------------------------------------------

Tantalus

Tantalus is an experimental object-relational mapping framework. Its 
purpose is to provide a mechanism for storing Smalltalk objects in a 
relational database as painlessly as possible, without having to embed SQL 
in the application logic. It allows an application to define how various 
classes are to be stored, and then provides methods for storing and 
retrieving objects.

more

Additional notes from the author: The difference between Tantalus and GLORP 
is the strategy used to ensure the integrity of the database. GLORP uses 
database transactions; Tantalus uses atomic updates.

----------------------------------------------------------------------------

SPrevayler

This is Prevayler (http://www.prevayler.com) for Squeak. Follow: SPrevalyer 
for more information.

----------------------------------------------------------------------------

See also ReferenceStream

----------------------------------------------------------------------------

Answers from the list (unedited) hjh -- 17-Feb-2003 Date: Sun, 16 Feb 2003 
From: Hannes Hirzel Subject: Implemeting a memory OO database in Squeak 
(was Re: port of prevayler) To: The general-purpose Squeak developers list

Hi Frank
Frank Sergeant wrote:
> Marco Paga wrote:

> > Is anybody out there who wants to port prevyler to squeak? I have > > a 
version aviable, but that one is badly designed. You can't

> >it in any way.
> > So the next version NEEDS to be better.
> > If you are interested please let me know.
>
> How is your implementation licensed? From the heading of your 13 Oct
> 2002 changeset, it looks like you are using GPL. If so, I think this is
> a terrible idea. If it is something for Squeak, why not use Squeak-L or
> something even more permissive? Is your implementation derived from the
> actual Prevayler code (which is LGPL)? I do not see how I can use GPL
> or LPGL.

Marco is planning to rewrite the whole thing. He likes to do it test-driven 
and is looking for somebody to work with him. (SUnit !)

As prevalyer is written in Java and Marco is writing it in Squeak there is 
no need to choose GPL and LPGL for the Squeak version. It is the code of 
Prevalyer in Java which is GPL licensed, not the ideas.

The ideas are straightforward and generally known. You may use them in any 
context and write software and put the software under any license you like.

Regards Hannes Hirzel

P.S. There are already some OO and non OO database efforts for Squeak see 
http://minnow.cc.gatech.edu/squeak/512 - Databases and Persistence

File Dictionary MinneStore GemSqueak Kats MySQL PostgreSQL GOODS ODBC 
SQLite SIXX - Smalltalk object XML serializer/deserializer BerkeleyDB 
Plugin Magma Glorp Tantalus SPrevayler

Did somebody do an evaluation of the concepts / quality of these 
implementations? If yes, please post remarks there.

----------------------------------------------------------------------------

Date: Sun, 16 Feb 2003 From: Avi Bryant

Subject: Re: Implemeting a memory OO database in Squeak (was Re: port of 
prevayler) To: The general-purpose Squeak developers list

On Sun, 16 Feb 2003, Hannes Hirzel wrote:

> P.S. There are already some OO and non OO database efforts for Squeak
> see http://minnow.cc.gatech.edu/squeak/512 - Databases and Persistence

> Did somebody do an evaluation of the concepts / quality of these
> implementations?
> If yes, please post remarks there.

Object Databases:

> - File Dictionary

[snip] (comments of Avi Bryant moved up)

> - MinneStore

[snip] (comments of Avi Bryant moved up)

> - Kats

Provides transactional semantics for in memory smalltalk objects. Uses a 
modified compiler to trap inst var reads/writes so that the same instance 
can look different to different processes. No persistence (but watch out 
for Stephen's Chango, which should integrate Kats with a transparent 
persistence layer using a modified VM).

> - GOODS

GOODS is a public domain distributed OODB written in C++. This is a not 
quite finished client for it - the protocol and serialization code is 
stable, but the transactional level stuff needs a fair bit of work. All GC, 
crash recovery, concurrency, locking, schema evolution, etc is handled 
nicely by the C++ server. Transparent storage of any object (but can 
provide more efficient storage with optional type info).

(If anyone has a strong interest in using this, I'd be happy to finish the 
client - particularly if there's money involved ;).

> - Magma

Pure Squeak client/server OODB. Somewhat immature (no class schema 
evolution, no transaction logging) and not yet tested in the Real World, 
but very promising. Allows transparent, transactional storage of any

Squeak objects.

Relational Databases:

> - MySQL

Port of Josh Miller's socket-level VW MySQL driver. Stable enough for 
production work (at least with Colin Putney's patches, not sure if these 
have been merged yet). Maintains type information.

> - PostgreSQL

Socket-level Postgres driver. Seems stable in recent releases. Doesn't yet 
maintain type info (all columns are retured as strings).

> - ODBC

Solid FFI interface to ODBC. Works on Linux (with unixodbc), Windows, and 
Mac OS X (with some effort - bug Colin for details). Maintains type info, 
and I've tried it successfully with drivers for MySQL, Postgresql, and 
SQLite.

> - SQLite

An FFI interface to the SQLite library (small, in-process, fast relational 
DB). Less overhead (particularly configuration overhead) than going through 
ODBC. Simple and stable.

Object Relational Mapping:

> - Glorp

Object/Relational mapping framework, based on TOPLink. Maps rows into 
objects, to-one relationships into references, to-many relationships into 
collections. Currently only works with PostgreSQL. Not in production use 
AFAIK, but has extensive tests and is probably stable.

> - Tantalus

Another O/R mapping layer. Designed specifically to be used with MySQL, so 
implements transactions on the client side instead of using DB 
transactions, but has also been used with Postgres and SQLite. Otherwise 
fairly similar to GLORP in philosophy. In production use, but still 
immature.

Both GLORP and Tantalus can be adapted to use Apple's EOModel files for 
mapping metadata.

Not Evaluated:

> - GemSqueak
> - SIXX - Smalltalk object XML serializer/deserializer
> - BerkeleyDB Plugin
> - SPrevayler

Date: Mon, 17 Feb 2003 01:36:44 +0100 From: tblanchard_at_mac.com Subject: Re: 
Implemeting a memory OO database in Squeak (was Re: port of prevayler) To: 
The general-purpose Squeak developers list reply-to: The general-purpose 
Squeak developers list

On Monday, February 17, 2003, at 12:44 AM, Avi Bryant wrote:
>> - PostgreSQL
>
> Socket-level Postgres driver. Seems stable in recent releases.
> Doesn't
> yet maintain type info (all columns are retured as strings).

This is no longer true - the latest pgsql client provided with glorp does 
return typed data (if you ask the row for typedData).

You can install new type conversion blocks into the connection object if 
you like. They're just single argument blocks keyed by type oid. Core types 
not supported by default are TimeStamp and Time with Time Zone (for the 
simple reason that squeak has no concept of time zones out of the box). If 
you use the TimeStampWithTimeZoneDatabase, on SqueakMap then its easy to 
add type converters for these.

> Object Relational Mapping:
>
>> - Glorp
>
> Object/Relational mapping framework, based on TOPLink. Maps rows into
> objects, to-one relationships into references, to-many relationships
> into
> collections. Currently only works with PostgreSQL. Not in production
> use
> AFAIK, but has extensive tests and is probably stable.

The version on squeakmap is still pretty immature. Its missing code to 
generate primary keys - works fine for reads but not writes. I have a 
changeset that fixes this. I'm waiting for Alan to finish his current 
release cycle to work with him to integrate the changes. The next version 
released should be pretty decent.

>
> Both GLORP and Tantalus can be adapted to use Apple's EOModel files for
> mapping metadata.

I have a set of extensions to glorp to support eomodel files. A few people 
have seen them but I would like more testing. email me if you want em. I'm 
giving them back to glorp as well.

Todd Blanchard

Date: Mon, 17 Feb 2003 09:56:55 -0500 From: "Stephen Pair" Subject: RE: 
Implemeting a memory OO database in Squeak (was Re: port of prevayler) To: 
"'The general-purpose Squeak developers list'" reply-to: The 
general-purpose Squeak developers list

> > Did somebody do an evaluation of the concepts / quality of these
> > implementations? If yes, please post remarks there.
>
> ---
> Object Databases:

...snip...

> > - BerkeleyDB Plugin

The BDB plugin simply provides an interface to BerkeleyDB. It currently 
does not implement the entire BerkeleyDB API, but it gives you enough to be 
useful. Most of what's missing are APIs that are concerned with scalability 
and fault tolerance. What is there is stable and clean.