Record management - Apply a retention policy to a document

Context

SEC-17a-4 (17 CFR § 240.17a-4 - Records to be preserved by certain exchange members, brokers and dealers.) is a US regulatory related to the records preservation.

The main areas are related to secured storage, retention management, change and deletion prevention, legal hold, and audit trail.

 

Prerequisite

For the record documents storage, we will use Amazon S3 capabilities with a bucket with the following parameters:

• Versioning turned on
• Compliance mode turned on
• No default retention in the bucket (or default retention as 0)

cf. https://github.com/awsdocs/amazon-s3-developer-guide/blob/master/doc_source/object-lock-overview.md

cf. https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock.html

 

User stories

• As a granted user, I want to apply a retention policy to a given document
• As a broker dealer, I want to guarantee that a user can’t delete the record until the defined retention period has expired
• As a broker dealer, I want to guarantee that an administrator can’t delete the record until the defined retention period has expired
• As a broker dealer, I want to prevent the record deletion on the storage media
• As a broker dealer, I want to prevent the core metadata deletion
• As a broker dealer, I want to prevent the deletion of the index database referring to the record on the Nuxeo repository
• As a broker dealer, I want the event "Declared as a record" to be logged in the audit/history when I apply a retention policy to a document
• As a broker dealer, I want the event "Retention period started" to be logged in the audit/history including the expiration date when the retention starts
• As a broker dealer, I want the event "Retention period expired" to be logged in the audit/history once the retention period has ended
• As a user manager, I want to be able to configure who is allowed to apply a retention policy to a given document

 

Description

By using the retention module, it is already possible in Nuxeo Server to apply a retention policy to a document.

But, we need to improve the security around the existing feature.

Improvements:

• Apply the retention at storage media level by updating the expiration date (retain until date) on Amazon S3
  • Use setObjectRetention method (cf. https://docs.aws.amazon.com/AmazonS3/latest/API/Type_API_ObjectLockRetention.html )

• Prevent at Nuxeo level any user to delete or trash a document which is in retention period, even an administrator
• Add dedicated permissions for retention policy application (Set retention, Remove retention)
• Tag the document as a record

 

User experience

• Apply retention to a document:

 

• Apply retention to a list of documents (from search results page):

 

• Retention history display:

 

Acceptance criteria

• The expiration date is set at Amazon S3 level once the retention period starts,
• The expiration date defined on S3 is exactly the same one as on Nuxeo Server,
• As a user, I can NOT delete a document under retention,
• As an administrator, I can NOT delete a document under retention,
• As a user, I can delete a document once the retention has expired,
• As an administrator, I can delete a document once the retention has expired,
• I can apply a retention policy to a document only if I have the Set retention permission,
• As a developer, I can NOT delete a document under retention by using Nuxeo API,
• As a developer, I can delete a document once the retention has expired by using Nuxeo API,
• The event "Declared as a record" is displayed on the history of the document / Audit once I apply a retention policy
• The event "Retention period started" with the relevant expiration date is displayed on the history of the document / Audit once I apply a fixed time retention with no delay policy
• The event "Retention period started" with the relevant expiration date (=end of delay+retention period) is displayed on the history of the document / Audit once the delay expired when  I apply a fixed time retention with a delay
• The event "Retention period expired" is displayed on the history of the document / Audit once the expiration date is reached
• When I apply an event-based retention policy to a document:
  • it is put in hold at Amazon S3 level,
  • When the predefined event occurs, the expiration date is set at S3 level,
  • When the predefined event occurs, the temporary hold is removed,
• I can apply a legal hold to a record on event-based retention when the retention has not started,
• When I removed a legal hold on a record waiting for the retention to start, it does NOT remove the legal hold at S3 level,
• When I apply a legal hold and then remove it on a record with the retention period started, it does remove the legal hold at S3 level,
• The event "Retention period started" with the relevant expiration date (=event date+retention period) is displayed on the history of the document / Audit once the event occurs
• The event "Retention period expired" is displayed on the history of the document / Audit once the expiration date is reached

 

Sequencing diagram

 cf. attachment SEC17a_4_seq_apply-retention.png