Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-13587

Better manage Version creation vs Async processing

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 5.8
    • Fix Version/s: 5.8.0-HF07, 5.9.2
    • Component/s: Core
    • Tags:
    • Upgrade notes:
      Hide

      Specific asynchronous listeners that need to update versions can now do:

      doc.putContextData(CoreSession.ALLOW_VERSION_WRITE, Boolean.TRUE);
      session.saveDocument(doc);

      Where doc is a version.

      Show
      Specific asynchronous listeners that need to update versions can now do: doc.putContextData(CoreSession.ALLOW_VERSION_WRITE, Boolean.TRUE); session.saveDocument(doc); Where doc is a version.

      Description

      The problem

      We have some automatic processing (i.e. listeners) that can be fired when a Document is modified.

      Some of these processing can be heavy (ex : video conversion) so they are run asynchronously.

      The issue is that these async processing can mess up with version system.

      The typically reproduction case is :

      • update a Document (modifying a blob) and ask for version creation
      • Nuxeo will
      • create the version
      • start the async job

      Here, we typically have 2 problems :

      Stale data

      At the time the version is created, it contains not up to date data.

      Typically the video conversion will match the previous Blob, not the one actually inside the version

      Versions are read only anyway

      When the async listener will run, it can not update the version document since it is read only

      Possible action

      Step 1 : Avoid stale data

      Anyway, we must avoid stale data : even for other use cases (i.e. without any versioning issue), we should synchronously put all the computed fields to "blank".

      Typically, this makes sense that the live video doc is diplayed with a blank video storyboard : this is better that displaying the storyboard of the previous video.

      Step 2 : updating the version

      Here we actually have 3 options :

      Don't update the version (option A)

      We can consider that version should not store the computed fields or that if you want them : the user will have to wait until the async processing are completed before creating a version.

      In a sense it means : don't create a version until the document is complete, because anyway, otherwise you don't know what you are exactly versioning.

      Add API to allow updating the version (option B)

      An other option is to add a new API that listeners could use to "enable version modification".

      This is a way to solve the problem, but actually this does introduce some side effects :

      • using this new API, pretty much any code will be able to modify version : this is bad !
      • if I using something like a digest/signature on versions : it will actually change depending at what time I compute the digest on the version

      Make version temporary (option C)

      Ideally, the version that was just created should be marked as "temporary / still writable" until all associated async processing are done.
      Then when all processing is done, system should remove the "writable" flag and the versions become really immutable.

      In a sense, this is a variant of the previous option : but cleaner.

      As a trade off, this puts much more pressure on the infrastructure, since determine at what point all async listeners associated to a version creation are completed is not that easy (especially in a distributed / cluster environment)

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  PagerDuty

                  Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.