-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 9.10, 10.10
-
Fix Version/s: 9.10-HF33, 10.10-HF08, 11.1, 2021.0
-
Component/s: Renditions
Requesting a stored rendition while it is currently being built causes Nuxeo to use the rendition in the transient store that was generated from a prior rendition request - and store it as the current rendition, while simultaneously generating the new rendition into the transient store - but failing to deliver or store the new rendition. The result is that delivered renditions are perpetually one iteration behind.
Reproduction steps on Nuxeo 9.10-HF30 with only JSF UI installed:
1. Add test-lazy-rendition-contribution to the "custom" template without rendition definitions "iamlazy", "lazyAutomation" and "lazyDelayedErrorAutomationRendition".
cd $NUXEO_HOME/templates/custom
mkdir config
wget https://raw.githubusercontent.com/nuxeo/nuxeo/9.10/nuxeo-features/nuxeo-platform-rendition/nuxeo-platform-rendition-core/src/test/resources/test-lazy-rendition-contrib.xml
mv test-lazy-rendition-contrib.xml test-lazy-rendition-config.xml
# Manually remove rendition definitions "iamlazy", "lazyAutomation" and "lazyDelayedErrorAutomationRendition" from file "test-lazy-rendition-config.xml"
2. Add "custom" to templates in nuxeo.conf
3. Create workspace1
curl -F "cmisaction=createFolder" -F "propertyId[0]=cmis:objectTypeId" -F "propertyValue[0]=Workspace" -F "propertyId[1]=cmis:name" -F "propertyValue[1]=workspace1" -u Administrator:Administrator "http://localhost:8080/nuxeo/json/cmis/default/root/default-domain/workspaces"
4. Create folder1 in workspace1
issued="1340000000000"; curl -F "cmisaction=createFolder" -F "propertyId[0]=cmis:objectTypeId" -F "propertyValue[0]=Folder" -F "propertyId[1]=cmis:name" -F "propertyValue[1]=folder1" -F "propertyId[2]=dc:issued" -F "propertyValue[2]=${issued}" -u Administrator:Administrator "http://localhost:8080/nuxeo/json/cmis/default/root/default-domain/workspaces/workspace1"
5. Upload ten files into folder1, each of which has a Main File size of approximately 50MB. The objective is to make the folder size sufficiently large to cause the zipTreeExportLazily rendition to take several seconds to generate (will vary, of course, based on test server configuration)
for i in {1..10}; do dd if=/dev/urandom of=file$i count=102400 && curl -F "cmisaction=createDocument" -F "propertyId[0]=cmis:objectTypeId" -F "propertyValue[0]=File" -F "propertyId[1]=cmis:name" -F "propertyValue[1]=file${i}" -F "content=@file${i}" -u Administrator:Administrator "http://localhost:8080/nuxeo/json/cmis/default/root/default-domain/workspaces/workspace1/folder1"; done
6. Generate a zipTreeExportLazily stored rendition by executing the following script
#!/bin/bash rm -f inprogress export.zip while : ; do result=$(curl -s -D- -OJ -u Administrator:Administrator "http://localhost:8080/nuxeo/json/cmis/default/root/default-domain/workspaces/workspace1/folder1?cmisselector=content&streamId=nuxeo:rendition:zipTreeExportLazily") if [[ "$result" == *"Content-Type: application/zip"* ]]; then break fi done
7. Verify the downloaded "export.zip" file matches the folder contents
8. Add a new file to folder1 by incrementing i each time starting with 11
i=11; dd if=/dev/urandom of=file$i count=102400 && curl -F "cmisaction=createDocument" -F "propertyId[0]=cmis:objectTypeId" -F "propertyValue[0]=File" -F "propertyId[1]=cmis:name" -F "propertyValue[1]=file${i}" -F "content=@file${i}" -u Administrator:Administrator "http://localhost:8080/nuxeo/json/cmis/default/root/default-domain/workspaces/workspace1/folder1"
9. Increment folder1 property dc:issued value (propertyValue[0]=<previous value + 1> below starting with 1340000000001) so its rendition will be re-generated on next request
issued="1340000000001"; curl -F "cmisaction=update" -F "propertyId[0]=dc:issued" -F "propertyValue[0]=${issued}" -u Administrator:Administrator "http://localhost:8080/nuxeo/json/cmis/default/root/default-domain/workspaces/workspace1/folder1"
10. Re-execute step #6
11. Inspect the downloaded "export.zip" file and observe that the file added in step 8 is absent
12. Repeat steps 8-11 and observe each time that the downloaded zip is one folder revision behind
I believe the problem is caused by the fact that a Lazy Rendition that is both marked "Completed" and has a source change should be reset to "Not Completed" right before a Work to regenerate the rendition is scheduled. See lines https://github.com/nuxeo/nuxeo/blob/bb1b6344ac3c5be08eea557e618fd8f0808650ff/nuxeo-features/nuxeo-platform-rendition/nuxeo-platform-rendition-core/src/main/java/org/nuxeo/ecm/platform/rendition/lazy/AbstractLazyCachableRenditionProvider.java#L224 and https://github.com/nuxeo/nuxeo/blob/bb1b6344ac3c5be08eea557e618fd8f0808650ff/nuxeo-features/nuxeo-platform-rendition/nuxeo-platform-rendition-core/src/main/java/org/nuxeo/ecm/platform/rendition/lazy/AbstractLazyCachableRenditionProvider.java#L251.
