Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-25601

Cannot import pictures with dimension over 65500

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 9.10, 10.2
    • Fix Version/s: 9.10-HF23, 10.3
    • Component/s: Convert
    • Release Notes Summary:
      Pictures with any dimensions can be imported.
    • Tags:
    • Backlog priority:
      1,100
    • Sprint:
      nxcore 10.10.1
    • Story Points:
      5

      Description

      Issue description

      1. install nuxeo
      2. install DAM addon
      3. create a Picture document
      4. uncompress bigtif.zip file
      5. add the attached TIFF file (90000x400) named Test_90000_400_raw.tif
      6. observe the following error in the logs
        2018-08-17 14:50:40,095 ERROR [Nuxeo-Work-pictureViewsGeneration-1:default:fef952c9-15e2-4481-8216-4490c84f7b22:file:content:pictureView] [org.nuxeo.ecm.platform.picture.core.im.IMImageUtils] ImageMagick failed on command: resizer
        org.nuxeo.ecm.platform.commandline.executor.api.CommandException: Error code 1 return by command: convert -define registry:temporary-path=#{nuxeo.tmp.dir} -quiet -resize #{targetWidth}x#{targetHeight}  #{inputFilePath}[0] png:- | convert - -background white -flatten -depth #{targetDepth} #{outputFilePath}
        convert: Maximum supported image dimension is 65500 pixels `/Users/ffischer/nuxeo-server-9.10-tomcat/tmp/nuxeoImageTarget1105635962525742549.jpg' @ error/jpeg.c/JPEGErrorHandler/332.
        	at org.nuxeo.ecm.platform.commandline.executor.api.ExecResult.<init>(ExecResult.java:62)
        	at org.nuxeo.ecm.platform.commandline.executor.service.executors.ShellExecutor.exec(ShellExecutor.java:83)
        	at org.nuxeo.ecm.platform.commandline.executor.service.CommandLineExecutorComponent.execCommand(CommandLineExecutorComponent.java:173)
        	at org.nuxeo.ecm.platform.picture.magick.utils.ImageResizer.resize(ImageResizer.java:60)
        	at org.nuxeo.ecm.platform.picture.core.im.IMImageUtils$2.callImageMagick(IMImageUtils.java:144)
        	at org.nuxeo.ecm.platform.picture.core.im.IMImageUtils$ImageMagickCaller.call(IMImageUtils.java:66)
        	at org.nuxeo.ecm.platform.picture.core.im.IMImageUtils.resize(IMImageUtils.java:146)
        	at org.nuxeo.ecm.platform.picture.ImagingComponent.resize(ImagingComponent.java:111)
        	at org.nuxeo.ecm.platform.picture.convert.ResizePictureConverter.convert(ResizePictureConverter.java:65)
        	at org.nuxeo.ecm.core.convert.service.ConversionServiceImpl.convert(ConversionServiceImpl.java:322)
        	at org.nuxeo.ecm.automation.core.operations.blob.RunConverter.run(RunConverter.java:66)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        	at org.nuxeo.ecm.automation.core.impl.InvokableMethod.doInvoke(InvokableMethod.java:173)
        	at org.nuxeo.ecm.automation.core.impl.InvokableMethod.invoke(InvokableMethod.java:178)
        	at org.nuxeo.ecm.automation.core.impl.OperationChainCompiler$OperationMethod.invoke(OperationChainCompiler.java:151)
        	at org.nuxeo.ecm.automation.core.impl.OperationChainCompiler$CompiledChainImpl.lambda$invoke$0(OperationChainCompiler.java:218)
        	at org.nuxeo.ecm.automation.OperationContext.call(OperationContext.java:328)
        	at org.nuxeo.ecm.automation.OperationContext.callWithChainParameters(OperationContext.java:293)
        	at org.nuxeo.ecm.automation.core.impl.OperationChainCompiler$CompiledChainImpl.invoke(OperationChainCompiler.java:215)
        	at org.nuxeo.ecm.automation.core.impl.OperationServiceImpl.run(OperationServiceImpl.java:115)
        	at org.nuxeo.ecm.automation.core.impl.OperationServiceImpl.lambda$run$0(OperationServiceImpl.java:105)
        	at org.nuxeo.ecm.automation.OperationContext.call(OperationContext.java:328)
        	at org.nuxeo.ecm.automation.OperationContext.callWithChainParameters(OperationContext.java:293)
        	at org.nuxeo.ecm.automation.core.impl.OperationServiceImpl.run(OperationServiceImpl.java:105)
        	at org.nuxeo.ecm.platform.picture.ImagingComponent.callPictureConversionChain(ImagingComponent.java:454)
        	at org.nuxeo.ecm.platform.picture.ImagingComponent.computeView(ImagingComponent.java:400)
        	at org.nuxeo.ecm.platform.picture.ImagingComponent.computeView(ImagingComponent.java:280)
        	at org.nuxeo.ecm.platform.picture.ImagingComponent.computeViewsFor(ImagingComponent.java:476)
        	at org.nuxeo.ecm.platform.picture.api.adapters.AbstractPictureAdapter.addViews(AbstractPictureAdapter.java:295)
        	at org.nuxeo.ecm.platform.picture.api.adapters.DefaultPictureAdapter.fillPictureViews(DefaultPictureAdapter.java:98)
        	at org.nuxeo.ecm.platform.picture.PictureViewsGenerationWork.work(PictureViewsGenerationWork.java:101)
        	at org.nuxeo.ecm.core.work.AbstractWork.runWorkWithTransaction(AbstractWork.java:435)
        	at org.nuxeo.ecm.core.work.AbstractWork.run(AbstractWork.java:355)
        	at org.nuxeo.ecm.core.work.WorkHolder.run(WorkHolder.java:57)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        	at java.lang.Thread.run(Thread.java:745)
        

      Primary analysis

      1. This is due to the fact JPEG files have only 16bits per dimension and thus are limited to 65500x65500.
        https://www.w3.org/Graphics/JPEG/jfif3.pdf
        https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
      2. The current conversions are handled by the following resizer:
        https://github.com/nuxeo/nuxeo/blob/9.10/nuxeo-features/nuxeo-platform-imaging/nuxeo-platform-imaging-core/src/main/resources/OSGI-INF/commandline-imagemagick-contrib.xml#L23
      3. Which is overriden during processing here:
        https://github.com/nuxeo/nuxeo/blob/9.10/nuxeo-features/nuxeo-platform-imaging/nuxeo-platform-imaging-core/src/main/java/org/nuxeo/ecm/platform/picture/magick/utils/ImageResizer.java#L57
      4. And thus it uses the following resizer very similar to the first one:
        https://github.com/nuxeo/nuxeo/blob/9.10/nuxeo-features/nuxeo-platform-imaging/nuxeo-platform-imaging-core/src/main/resources/OSGI-INF/commandline-imagemagick-contrib.xml#L36

      Further testing

      1. based on the original contribution, the following command-line succeeds in converting the big TIFF into a JPEG:
        convert -quiet -resize 480 Test_90000_400_raw.tif[0] - | convert - -background white -flatten Test_90000_400_raw.jpg
        

        but modifying the contribution accordingdly (resizer and jpegResizer) is not enough and ends up with:

        2018-08-17 15:05:28,362 ERROR [Nuxeo-Work-pictureViewsGeneration-1:default:a90f90d9-aaa1-40be-93e9-05cc6314c8d7:file:content:pictureView] [org.nuxeo.ecm.platform.picture.core.im.IMImageUtils] ImageMagick failed on command: resizer
        org.nuxeo.ecm.platform.commandline.executor.api.CommandException: Error code 1 return by command: convert -define registry:temporary-path=#{nuxeo.tmp.dir} -quiet -background white -flatten -depth #{targetDepth} -resize #{targetWidth}x#{targetHeight}  #{inputFilePath}[0] - | convert - #{outputFilePath}
        convert: Maximum supported image dimension is 65500 pixels `/Users/ffischer/nuxeo-server-9.10-tomcat/tmp/nuxeoImageTarget3575702264766862299.jpg' @ error/jpeg.c/JPEGErrorHandler/332.
        	at org.nuxeo.ecm.platform.commandline.executor.api.ExecResult.<init>(ExecResult.java:62)
        	at org.nuxeo.ecm.platform.commandline.executor.service.executors.ShellExecutor.exec(ShellExecutor.java:83)
        	at org.nuxeo.ecm.platform.commandline.executor.service.CommandLineExecutorComponent.execCommand(CommandLineExecutorComponent.java:173)
        	at org.nuxeo.ecm.platform.picture.magick.utils.ImageResizer.resize(ImageResizer.java:60)
        

        This leads to think at some point the image is converted to JPG without being resized. Which is confirmed by doing a debugging session showing the following parameters:

        [convert, -define, registry:temporary-path=/Users/ffischer/nuxeo-server-9.10-tomcat/tmp, -quiet, -background, white, -flatten, -depth, 8, -resize, 90000x400, /Users/ffischer/nuxeo-server-9.10-tomcat/tmp/nuxeoImageSource322605250909450176.tif[0], -]
        

        90000 is still not supported by JPEG and ends up with the aforementioned error.
        This may indicate there is a first conversion done using the same dimensions as input and output.

      Expected behavior: big files should be supported and there should be no jpg format involved until the resizing (at the very least) is done.

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day, 2 hours
                1d 2h