Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-29769

Improve how list elements are added concurrently on DBS

    XMLWordPrintable

    Details

    • Release Notes Summary:
      Better concurrency management when adding a value to an empty list.
    • Tags:
    • Backlog priority:
      900
    • Sprint:
      nxFG 11.3.1

      Description

      After uploading a video which triggers several conversions, I've observed the following MongoDB commands

      date host thread command
      2020-10-14T14:31:26.783Z saig-preprod-nuxw-255-10-8-8-34-use Nuxeo-Work-videoConversion-1:default:ac255e5f-c25f-43e1-ab95-b87ab81d2a95:videoconv:SAI 424 MongoDB: UPDATE ac255e5f-c25f-43e1-ab95-b87ab81d2a95: Document{{$set=Document{{dc:modified=Wed Oct 14 14:31:26 UTC 2020, ecm:fulltextJobId=ac255e5f-c25f-43e1-ab95-b87ab81d2a95, vid:transcodedVideos=Document{{name=SAI 424, content=Document{{data=f1317184dcd9a1a92707e90f3d62b124, name=PAel01aERE_eng_424.mp4, mime-type=video/mp4, length=2152334}}, info=Document{{duration=40.6, frameRate=15.0, streams=[Document{{codec=h264 (High) (avc1 / 0x31637661), bitRate=360.0, streamInfo=Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 544x306, 360 kb/s, 15 fps, 15 tbr, 15360 tbn, 30 tbc (default), type=Video}}, Document{{codec=aac (LC) (mp4a / 0x6134706D), bitRate=59.0, streamInfo=Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 59 kb/s (default), type=Audio}}], width=544, format=mov,mp4,m4a,3gp,3g2,mj2, height=306}}}}}}, $inc=Document{{ecm:systemChangeToken=1}}}}
      2020-10-14T14:31:26.822Z saig-preprod-nuxw-255-10-8-12-163-use Nuxeo-Work-videoConversion-1:default:ac255e5f-c25f-43e1-ab95-b87ab81d2a95:videoconv:SAI 220 MongoDB: UPDATE ac255e5f-c25f-43e1-ab95-b87ab81d2a95: Document{{$set=Document{{dc:modified=Wed Oct 14 14:31:26 UTC 2020, ecm:fulltextJobId=ac255e5f-c25f-43e1-ab95-b87ab81d2a95, vid:transcodedVideos=Document{{name=SAI 220, content=Document{{data=24662f247a46468fa8131ef6a968f792, name=PAel01aERE_eng_220.mp4, mime-type=video/mp4, length=1254849}}, info=Document{{duration=40.6, frameRate=15.0, streams=[Document{{codec=h264 (High) (avc1 / 0x31637661), bitRate=183.0, streamInfo=Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 544x306, 183 kb/s, 15 fps, 15 tbr, 15360 tbn, 30 tbc (default), type=Video}}, Document{{codec=aac (LC) (mp4a / 0x6134706D), bitRate=59.0, streamInfo=Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 59 kb/s (default), type=Audio}}], width=544, format=mov,mp4,m4a,3gp,3g2,mj2, height=306}}}}}}, $inc=Document{{ecm:systemChangeToken=1}}}}
      2020-10-14T14:31:54.723Z saig-preprod-nuxw-255-10-8-12-163-use Nuxeo-Work-videoConversion-1:default:ac255e5f-c25f-43e1-ab95-b87ab81d2a95:videoconv:480p MongoDB: UPDATE ac255e5f-c25f-43e1-ab95-b87ab81d2a95: Document{{$push=Document{{vid:transcodedVideos=Document{{name=480p, content=Document{{data=0f2dbb9e85e6c39c5c2a296c7806c0b3, name=PAel01aERE_eng_480p.mp4, mime-type=video/mp4, length=2652963}}, info=Document{{duration=40.56, frameRate=59.94, streams=Document{{codec=h264 (High) (avc1 / 0x31637661), bitRate=402.0, streamInfo=Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 854x480, 402 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 119.88 tbc (default), type=Video}}, Document{{codec=aac (LC) (mp4a / 0x6134706D), bitRate=108.0, streamInfo=Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 108 kb/s (default), type=Audio}}, width=854, format=mov,mp4,m4a,3gp,3g2,mj2, height=480}}}}}}, $inc=Document{{ecm:systemChangeToken=1}}}}

      What is interesting is:

      • the first 2 lines are from 2 differents nodes
      • the first 2 lines correspond to dates very close
      • the first 2 lines both execute a $set MongoDB query whereas the 3rd line executes a $push query

      The consequence is that the conversion SAI 424 is overwritten by the conversion SAI 220

      Expected result: all conversions are correctly saved and no conversion is overwritten by another one

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day
                  1d

                    PagerDuty

                    Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.