[NXP-29769] Improve how list elements are added concurrently on DBS - Nuxeo Issue Tracker

XML

Word

Printable

Details

Type: Bug
Status: Resolved
Priority: Critical
Resolution: Fixed
Affects Version/s: 9.10, 10.10
Fix Version/s: 9.10-HF49, 10.10-HF36, 11.4, 2021.0
Component/s: Convert, Core, Core MongoDB

Release Notes Summary:
Better concurrency management when adding a value to an empty list.
Tags:
- SupCom
- nxFG
Backlog priority:
900
Sprint:
nxFG 11.3.1

Description

After uploading a video which triggers several conversions, I've observed the following MongoDB commands

date	host	thread	command
2020-10-14T14:31:26.783Z	saig-preprod-nuxw-255-10-8-8-34-use	Nuxeo-Work-videoConversion-1:default:ac255e5f-c25f-43e1-ab95-b87ab81d2a95:videoconv:SAI 424	MongoDB: UPDATE ac255e5f-c25f-43e1-ab95-b87ab81d2a95: Document{{$set=Document{{dc:modified=Wed Oct 14 14:31:26 UTC 2020, ecm:fulltextJobId=ac255e5f-c25f-43e1-ab95-b87ab81d2a95, vid:transcodedVideos=Document{{name=SAI 424, content=Document{{data=f1317184dcd9a1a92707e90f3d62b124, name=PAel01aERE_eng_424.mp4, mime-type=video/mp4, length=2152334}}, info=Document{{duration=40.6, frameRate=15.0, streams=[Document{{codec=h264 (High) (avc1 / 0x31637661), bitRate=360.0, streamInfo=Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 544x306, 360 kb/s, 15 fps, 15 tbr, 15360 tbn, 30 tbc (default), type=Video}}, Document{{codec=aac (LC) (mp4a / 0x6134706D), bitRate=59.0, streamInfo=Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 59 kb/s (default), type=Audio}}], width=544, format=mov,mp4,m4a,3gp,3g2,mj2, height=306}}}}}}, $inc=Document{{ecm:systemChangeToken=1}}}}
2020-10-14T14:31:26.822Z	saig-preprod-nuxw-255-10-8-12-163-use	Nuxeo-Work-videoConversion-1:default:ac255e5f-c25f-43e1-ab95-b87ab81d2a95:videoconv:SAI 220	MongoDB: UPDATE ac255e5f-c25f-43e1-ab95-b87ab81d2a95: Document{{$set=Document{{dc:modified=Wed Oct 14 14:31:26 UTC 2020, ecm:fulltextJobId=ac255e5f-c25f-43e1-ab95-b87ab81d2a95, vid:transcodedVideos=Document{{name=SAI 220, content=Document{{data=24662f247a46468fa8131ef6a968f792, name=PAel01aERE_eng_220.mp4, mime-type=video/mp4, length=1254849}}, info=Document{{duration=40.6, frameRate=15.0, streams=[Document{{codec=h264 (High) (avc1 / 0x31637661), bitRate=183.0, streamInfo=Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 544x306, 183 kb/s, 15 fps, 15 tbr, 15360 tbn, 30 tbc (default), type=Video}}, Document{{codec=aac (LC) (mp4a / 0x6134706D), bitRate=59.0, streamInfo=Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 59 kb/s (default), type=Audio}}], width=544, format=mov,mp4,m4a,3gp,3g2,mj2, height=306}}}}}}, $inc=Document{{ecm:systemChangeToken=1}}}}
2020-10-14T14:31:54.723Z	saig-preprod-nuxw-255-10-8-12-163-use	Nuxeo-Work-videoConversion-1:default:ac255e5f-c25f-43e1-ab95-b87ab81d2a95:videoconv:480p	MongoDB: UPDATE ac255e5f-c25f-43e1-ab95-b87ab81d2a95: Document{{$push=Document{{vid:transcodedVideos=Document{{name=480p, content=Document{{data=0f2dbb9e85e6c39c5c2a296c7806c0b3, name=PAel01aERE_eng_480p.mp4, mime-type=video/mp4, length=2652963}}, info=Document{{duration=40.56, frameRate=59.94, streams=Document{{codec=h264 (High) (avc1 / 0x31637661), bitRate=402.0, streamInfo=Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 854x480, 402 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 119.88 tbc (default), type=Video}}, Document{{codec=aac (LC) (mp4a / 0x6134706D), bitRate=108.0, streamInfo=Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 108 kb/s (default), type=Audio}}, width=854, format=mov,mp4,m4a,3gp,3g2,mj2, height=480}}}}}}, $inc=Document{{ecm:systemChangeToken=1}}}}

What is interesting is:

the first 2 lines are from 2 differents nodes
the first 2 lines correspond to dates very close
the first 2 lines both execute a $set MongoDB query whereas the 3rd line executes a $push query

The consequence is that the conversion SAI 424 is overwritten by the conversion SAI 220

Expected result: all conversions are correctly saved and no conversion is overwritten by another one