Right now it is possible to write Axiom applications that are concurrent. However, they will be subtly broken, as the cache will continue to keep values around which are potentially incorrect. The clearest problem is that if you do something like, for example:
class Account(Item):
balance = money(default=0)
class Entry(Item):
account = reference(allowNone=False)
amount = money(allowNone=False)
def __init__(self, **kw):
super(Entry, self).__init__(**kw)
self.account.balance += self.amount
This is perfectly safe if only one process will ever create Entry objects in your Store at a time. However, it becomes a problem when multiple processes do, because a cached value for account.balance would make a subprocess generate an incorrect running total if another process were simultaneously creating another Entry.
While this doesn't deal with every *possible* issue (for example: methods which are supposed to call network methods have to happen in the correct process) it should at least deal with consistency of data.
This is also specific to the sqlite backend, should other backends emerge. Database concurrency interacts weirdly with performance and you have to understand how that should work. For now, my concern is to make concurrency 100% safe, at least from the perspective of data integrity.
The proposal is as follows:
- Create a table with the following schema (since this is so low level I don't think it should be an Item):
CREATE TABLE axiom_transaction (txn_id INTEGER);
INSERT INTO axiom_transaction VALUES (0);
- Give every Store object a lastTransactionID so that it knows what the last transaction it performed was.
- When every transaction starts, issue some SQL like this:
BEGIN IMMEDIATE TRANSACTION;
UPDATE axiom_transaction SET txn_id = txn_id + 1;
SELECT txn_id FROM axiom_transaction;
- If txn_id is not the expected value, drop all cached attributes currently in memory. Objects in memory may remain in memory and may retain inmemory() attributes.
- Once cached attribute values have been dropped, Items must have the ability to "cache fault" and pick up new values from the database by issuing the appropriate getItemByID.
If this gets implemented, we should examine the performance impact. If it is measurable (I suspect it will not be, since the only expensive operation happens when there is lots of concurrent access, which there should not be) then we can simply do all the checking on open, by keeping pidfiles around for every process that opens the database rather than just axiomatic processes, and refusing to open a database that another live process has open which isn't explicitly configured for concurrent access.
It's worth noting that concurrent access for operations like the one listed above remains unsafe if you are not using transactions, but oh my god what is wrong with you start using transactions already. Your app will go 10x faster, too.
