Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-30382

Tune recomputeThumbnail bulk action to avoid transaction timeout or stream failure

    XMLWordPrintable

    Details

    • Release Notes Summary:
      recomputeThumbnail bulk action is tuned to avoid transaction timeout or stream failure
    • Team:
      PLATFORM
    • Sprint:
      nxplatform #36, nxplatform #37
    • Story Points:
      1

      Description

      Today the recomputeThumnail action is configured like this

         <action name="recomputeThumbnails" inputStream="bulk/recomputeThumbnails" bucketSize="100" batchSize="10"
            httpEnabled="false" />
      ...
            <policy name="default" maxRetries="3" delay="1s" maxDelay="10s" continueOnFailure="true" />
      

      this means 100 documents per records with batch of 10 docs per transaction
      The transaction nuxeo.db.transactiontimeout is 5min by default, if thumbnail conversions cannot be done within 5min there is a transaction timeout that triggers a retry so the record is reprocessed, potentially this will generate again a timeout while processing the same batch of documents.

      At best the record is skipped and we miss some conversions, at worst we failed to respect the kafka poll interval and this creates a stream failure which requires a restart of the node.

      possible solution

      • assume that the processing to generate the thumbnail is slow and reduce the batch size to 1
      • in the case of a transaction timeout make sure that the bulk command status is completed with an error status.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: