Better manage Version creation vs Async processing

The problem

We have some automatic processing (i.e. listeners) that can be fired when a Document is modified.

Some of these processing can be heavy (ex : video conversion) so they are run asynchronously.

The issue is that these async processing can mess up with version system.

The typically reproduction case is :

• update a Document (modifying a blob) and ask for version creation
• Nuxeo will
• create the version
• start the async job

Here, we typically have 2 problems :

Stale data

At the time the version is created, it contains not up to date data.

Typically the video conversion will match the previous Blob, not the one actually inside the version

Versions are read only anyway

When the async listener will run, it can not update the version document since it is read only

Possible action

Step 1 : Avoid stale data

Anyway, we must avoid stale data : even for other use cases (i.e. without any versioning issue), we should synchronously put all the computed fields to "blank".

Typically, this makes sense that the live video doc is diplayed with a blank video storyboard : this is better that displaying the storyboard of the previous video.

Step 2 : updating the version

Here we actually have 3 options :

Don't update the version (option A)

We can consider that version should not store the computed fields or that if you want them : the user will have to wait until the async processing are completed before creating a version.

In a sense it means : don't create a version until the document is complete, because anyway, otherwise you don't know what you are exactly versioning.

Add API to allow updating the version (option B)

An other option is to add a new API that listeners could use to "enable version modification".

This is a way to solve the problem, but actually this does introduce some side effects :

• using this new API, pretty much any code will be able to modify version : this is bad !
• if I using something like a digest/signature on versions : it will actually change depending at what time I compute the digest on the version

Make version temporary (option C)

Ideally, the version that was just created should be marked as "temporary / still writable" until all associated async processing are done.
Then when all processing is done, system should remove the "writable" flag and the versions become really immutable.

In a sense, this is a variant of the previous option : but cleaner.

As a trade off, this puts much more pressure on the infrastructure, since determine at what point all async listeners associated to a version creation are completed is not that easy (especially in a distributed / cluster environment)