Record management - Handle event-based retention

Context

SEC-17a-4 (17 CFR § 240.17a-4 - Records to be preserved by certain exchange members, brokers and dealers.) is a US regulatory related to the records preservation.

The main areas are related to secured storage, retention management, change and deletion prevention, legal hold, and audit trail.

Prerequisite

For the record documents storage, we will use Amazon S3 capabilities with a bucket with the following parameters:

• Versioning turned on
• Compliance mode turned on
• No default retention in the bucket (or default retention as 0)

cf. https://github.com/awsdocs/amazon-s3-developer-guide/blob/master/doc_source/object-lock-overview.md

cf. https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock.html

Definition

An event based retention allows to define a retention period to start after a defined event. Most of the time, the date of the event is unknown at time of retention application of the document.

The events can be very different depending on the organization, document type, region...
A list of real examples of event based retention are available here:
https://docs.google.com/spreadsheets/d/1TSVOQBTbUF_pE83d7y9jcYVLiDN1VJn0ou0B1PM6AiY/edit?usp=sharing 

Description

By using the retention module, I can define an event based retention policy, meaning that the retention will start once the predefined event occurs.

But:

• There is no lock of the record waiting for the retention to start
• There is no lock and retention period applied at Amazon S3 level
• The event configuration requires to use expression language (EL) which is not user friendly for a non developer user (most of the record managers don't have developer skills).
• In addition, it requires to use only internal events, which doesn't fit with all the use cases.

Improvements:

• When the document becomes a record:
  • Nuxeo stores the record on S3 bucket with compliance mode with no expiration date,
  • Then, Nuxeo automatically applies a legal hold,

• When the event occurs and we want to trigger the retention beginning:
  • Nuxeo removes the legal hold on S3,
  • Nuxeo updates the record by adding the expiration date,
• Add the capability define in a user friendly way the event in the retention rule 
• Add the capability to manually or automatically create an event, based on core or external events.

Warning

There is a use case to take care: if a legal hold is triggered and then removed before the retention to start, we must be careful to not removed the legal hold at S3 level.