Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-31146

Avoid OOM on nuxeo-diff-content addon while rendering huge diff

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.10-HF65, 2021.25
    • Component/s: Diff
    • Release Notes Summary:
      Limit diffs by counting them to avoid OOM errors in DaisyDiff
    • Tags:
    • Upgrade notes:
      Hide

      We introduced a new configuration for the nuxeo 10.10 and nuxeo-jsf-ui projects to limit comparisons by diff count to avoid an OOM (Out Of Memory) error.
      The default is fixed to 10 000 diffs per comparison and can be overriden like below (-1 means unlimitted):

      <?xml version="1.0"?>
<component name="org.nuxeo.ecm.diff.settings.test">
  <require>org.nuxeo.ecm.diff.settings</require>
  <extension target="org.nuxeo.runtime.ConfigurationService" point="configuration">
    <property name="nuxeo.diff.limit">-1</property>
  </extension>
</component>

      Reaching the diff count limit will not return a partial diff.
      It will throw a `[hidden.org](http://hidden.org).eclipse.core.runtime.AssertionFailedException`
      Which is DaisyDiff’s expected behaviour (the underlying diff library) when generating a comparison that is too big.

      Show
      We introduced a new configuration for the nuxeo 10.10 and nuxeo-jsf-ui projects to limit comparisons by diff count to avoid an OOM (Out Of Memory) error. The default is fixed to 10 000 diffs per comparison and can be overriden like below (-1 means unlimitted): <?xml version= "1.0" ?> <component name= "org.nuxeo.ecm.diff.settings.test" > <require> org.nuxeo.ecm.diff.settings </require> <extension target= "org.nuxeo.runtime.ConfigurationService" point= "configuration" > <property name= "nuxeo.diff.limit" > -1 </property> </extension> </component> Reaching the diff count limit will not return a partial diff. It will throw a ` [hidden.org] ( http://hidden.org).eclipse.core.runtime.AssertionFailedException ` Which is DaisyDiff’s expected behaviour (the underlying diff library) when generating a comparison that is too big.
    • Team:
      PLATFORM
    • Sprint:
      nxplatform #66, nxplatform #67
    • Story Points:
      5

      Description

      With huge binary change, we may saturate the heap with org.eclipse.compare.rangedifferencer.OldDifferencer$LinkedRangeDifference object.

      The number of diff should be limited we cannot accumulate an infinite number of diff objects.

      Stack trace:

      http-nio-0.0.0.0-8080-exec-433
  at org.eclipse.compare.rangedifferencer.OldDifferencer.findDifferences(Lhidden/org/eclipse/core/runtime/IProgressMonitor;Lorg/eclipse/compare/rangedifferencer/IRangeComparator;Lorg/eclipse/compare/rangedifferencer/IRangeComparator;)[Lorg/eclipse/compare/rangedifferencer/RangeDifference; (OldDifferencer.java:147)
  at org.eclipse.compare.rangedifferencer.RangeDifferencer.findDifferences(Lhidden/org/eclipse/core/runtime/IProgressMonitor;Lorg/eclipse/compare/internal/LCSSettings;Lorg/eclipse/compare/rangedifferencer/IRangeComparator;Lorg/eclipse/compare/rangedifferencer/IRangeComparator;)[Lorg/eclipse/compare/rangedifferencer/RangeDifference; (RangeDifferencer.java:108)
  at org.eclipse.compare.rangedifferencer.RangeDifferencer.findDifferences(Lorg/eclipse/compare/internal/LCSSettings;Lorg/eclipse/compare/rangedifferencer/IRangeComparator;Lorg/eclipse/compare/rangedifferencer/IRangeComparator;)[Lorg/eclipse/compare/rangedifferencer/RangeDifference; (RangeDifferencer.java:69)
  at org.outerj.daisy.diff.html.HTMLDiffer.diff(Lorg/outerj/daisy/diff/html/TextNodeComparator;Lorg/outerj/daisy/diff/html/TextNodeComparator;)V (HTMLDiffer.java:121)
  at org.nuxeo.ecm.diff.content.adapter.HtmlContentDiffer.getContentDiff(Lorg/nuxeo/ecm/core/api/Blob;Lorg/nuxeo/ecm/core/api/Blob;Ljava/util/Locale;)Ljava/util/List; (HtmlContentDiffer.java:140)
  at org.nuxeo.ecm.diff.content.adapter.base.ConverterBasedContentDiffAdapter.getContentDiffBlobs(Lorg/nuxeo/ecm/core/api/DocumentModel;Ljava/lang/String;Lorg/nuxeo/ecm/diff/content/adapter/base/ContentDiffConversionType;Ljava/util/Locale;)Ljava/util/List; (ConverterBasedContentDiffAdapter.java:139)
  at org.nuxeo.ecm.diff.content.adapter.base.ConverterBasedContentDiffAdapter.getContentDiffBlobs(Lorg/nuxeo/ecm/core/api/DocumentModel;Lorg/nuxeo/ecm/diff/content/adapter/base/ContentDiffConversionType;Ljava/util/Locale;)Ljava/util/List; (ConverterBasedContentDiffAdapter.java:61)
  at org.nuxeo.ecm.diff.content.adapter.base.AbstractContentDiffAdapter.getFileContentDiffBlobs(Lorg/nuxeo/ecm/core/api/DocumentModel;Lorg/nuxeo/ecm/diff/content/adapter/base/ContentDiffConversionType;Ljava/util/Locale;)Ljava/util/List; (AbstractContentDiffAdapter.java:54)
  at org.nuxeo.ecm.diff.content.restlet.ContentDiffRestlet.initCachedContentDiffBlobs(Lorg/restlet/Response;Ljava/lang/String;Lorg/nuxeo/ecm/diff/content/adapter/base/ContentDiffConversionType;)Ljava/util/List; (ContentDiffRestlet.java:199)
  at org.nuxeo.ecm.diff.content.restlet.ContentDiffRestlet.handle(Lorg/restlet/Request;Lorg/restlet/Response;)V (ContentDiffRestlet.java:130)
  at org.nuxeo.ecm.platform.ui.web.restAPI.ThreadSafeRestletFilter.doHandle(Lorg/restlet/Request;Lorg/restlet/Response;)I (ThreadSafeRestletFilter.java:48)
  at org.restlet.routing.Filter.handle(Lorg/restlet/Request;Lorg/restlet/Response;)V (Filter.java:197)
  at org.restlet.routing.Filter.doHandle(Lorg/restlet/Request;Lorg/restlet/Response;)I (Filter.java:150)
  at org.restlet.routing.Filter.handle(Lorg/restlet/Request;Lorg/restlet/Response;)V (Filter.java:197)
  at org.restlet.routing.Router.doHandle(Lorg/restlet/Restlet;Lorg/restlet/Request;Lorg/restlet/Response;)V (Router.java:422)
  at org.restlet.routing.Router.handle(Lorg/restlet/Request;Lorg/restlet/Response;)V (Router.java:641)
  at org.restlet.ext.servlet.ServletAdapter.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (ServletAdapter.java:195)
  at org.nuxeo.ecm.platform.ui.web.restAPI.RestletServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (RestletServlet.java:116)
  at javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (HttpServlet.java:764)

        Attachments

          Issue Links

          causes

          Task - A task that needs to be done. NXP-31401 Release Nuxeo Diff 1.9.4 with daisydiff-1.2-NX5

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          depends on

          Improvement - An improvement or enhancement to an existing feature or task. VEND-53 Allow to limit the number of diffs in a comparison

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. VEND-54 Allow configuring the max number of diffs in a comparison

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. NXP-31486 Add an option to configure nuxeo-diff maximum changes limit

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: