Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-28502

Fix permanent deletion of folder containing a large number of documents in MongoDB

    XMLWordPrintable

    Details

    • Release Notes Summary:
      Permanent deletion of folder containing a large number of documents is correctly handled with in MongoDB.
    • Tags:
    • Backlog priority:
      600
    • Sprint:
      nxplatform 11.1.27, nxplatform 11.1.28, nxplatform 11.1.29
    • Story Points:
      1

      Description

      Permanently deleting a folder containing 400,000+ document fails with MongoDB, here is the stacktrace:

      Caused by: org.bson.BsonMaximumSizeExceededException: Document size of 19632955 is larger than maximum of 16793600.
	at org.bson.BsonBinaryWriter.validateSize(BsonBinaryWriter.java:418) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.BsonBinaryWriter.backpatchSize(BsonBinaryWriter.java:412) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.BsonBinaryWriter.doWriteEndArray(BsonBinaryWriter.java:153) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.AbstractBsonWriter.writeEndArray(AbstractBsonWriter.java:350) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.LevelCountingBsonWriter.writeEndArray(LevelCountingBsonWriter.java:78) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonArrayCodec.encode(BsonArrayCodec.java:85) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonArrayCodec.encode(BsonArrayCodec.java:37) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.writeValue(BsonDocumentCodec.java:136) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:115) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:41) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.configuration.LazyCodec.encode(LazyCodec.java:37) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.writeValue(BsonDocumentCodec.java:136) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:115) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:41) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.configuration.LazyCodec.encode(LazyCodec.java:37) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.writeValue(BsonDocumentCodec.java:136) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:115) ~[mongo-java-driver-3.8.1.jar:?]
	at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:41) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.RequestMessage.addDocument(RequestMessage.java:238) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.RequestMessage.addDocument(RequestMessage.java:188) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:136) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:57) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:244) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:444) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:200) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:123) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:242) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:233) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:136) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.FindOperation$1.call(FindOperation.java:701) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.FindOperation$1.call(FindOperation.java:695) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:457) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:401) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.FindOperation.execute(FindOperation.java:695) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.operation.FindOperation.execute(FindOperation.java:83) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:179) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:132) ~[mongo-java-driver-3.8.1.jar:?]
	at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:86) ~[mongo-java-driver-3.8.1.jar:?]
	at org.nuxeo.ecm.core.storage.mongodb.MongoDBRepository.stream(MongoDBRepository.java:547) ~[nuxeo-core-storage-mongodb-10.10-HF18.jar:?]
	at org.nuxeo.ecm.core.storage.mongodb.MongoDBRepository.stream(MongoDBRepository.java:519) ~[nuxeo-core-storage-mongodb-10.10-HF18.jar:?]
	at org.nuxeo.ecm.core.storage.mongodb.MongoDBRepository.findAll(MongoDBRepository.java:513) ~[nuxeo-core-storage-mongodb-10.10-HF18.jar:?]
	at org.nuxeo.ecm.core.storage.mongodb.MongoDBRepository.readStates(MongoDBRepository.java:335) ~[nuxeo-core-storage-mongodb-10.10-HF18.jar:?]
	at org.nuxeo.ecm.core.storage.dbs.DBSCachingRepository.readStates(DBSCachingRepository.java:203) ~[nuxeo-core-storage-dbs-10.10-HF17.jar:?]
	at org.nuxeo.ecm.core.storage.dbs.DBSTransactionState.getStatesForUpdate(DBSTransactionState.java:225) ~[nuxeo-core-storage-dbs-10.10-HF17.jar:?]
	at org.nuxeo.ecm.core.storage.dbs.DBSSession.getDocuments(DBSSession.java:422) ~[nuxeo-core-storage-dbs-10.10-HF17.jar:?]
	at org.nuxeo.ecm.core.storage.dbs.DBSSession.query(DBSSession.java:1503) ~[nuxeo-core-storage-dbs-10.10-HF17.jar:?]
	at org.nuxeo.ecm.core.storage.dbs.DBSSession.query(DBSSession.java:156) ~[nuxeo-core-storage-dbs-10.10-HF17.jar:?]
	at sun.reflect.GeneratedMethodAccessor127.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
	at org.nuxeo.ecm.core.storage.dbs.DBSRepositoryBase$DBSSessionInvoker.invoke(DBSRepositoryBase.java:492) ~[nuxeo-core-storage-dbs-10.10-HF17.jar:?]
	at com.sun.proxy.$Proxy69.query(Unknown Source) ~[?:?]
	at org.nuxeo.ecm.core.api.AbstractSession.query(AbstractSession.java:1177) ~[nuxeo-core-10.10-HF19.jar:?]
	at org.nuxeo.ecm.core.api.AbstractSession.query(AbstractSession.java:1110) ~[nuxeo-core-10.10-HF19.jar:?]
	at org.nuxeo.ecm.core.api.AbstractSession.query(AbstractSession.java:1103) ~[nuxeo-core-10.10-HF19.jar:?]
	at org.nuxeo.ecm.core.api.AbstractSession.query(AbstractSession.java:1083) ~[nuxeo-core-10.10-HF19.jar:?]
	at org.nuxeo.ecm.core.trash.AbstractTrashService.getDocuments(AbstractTrashService.java:287) ~[nuxeo-core-10.10-HF19.jar:?]
	at org.nuxeo.ecm.webapp.action.DeleteActionsBean.getCanEmptyTrash(DeleteActionsBean.java:144) ~[nuxeo-platform-webapp-core-10.10-HF05.jar:?]
	at sun.reflect.GeneratedMethodAccessor778.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
	at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.core.BijectionInterceptor.aroundInvoke(BijectionInterceptor.java:79) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.nuxeo.ecm.platform.ui.web.util.NuxeoExceptionInterceptor.aroundInvoke(NuxeoExceptionInterceptor.java:80) ~[nuxeo-platform-ui-web-10.10-HF15.jar:?]
	at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:196) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:114) ~[jboss-seam-2.3.1.Final.NX03.jar:2.3.1.Final.NX03-SNAPSHOT]
	at org.nuxeo.ecm.webapp.action.DeleteActionsBean_$$_javassist_seam_51.getCanEmptyTrash(DeleteActionsBean_$$_javassist_seam_51.java) ~[nuxeo-platform-webapp-core-10.10-HF05.jar:?]
	at sun.reflect.GeneratedMethodAccessor777.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
	at javax.el.BeanELResolver.getValue(BeanELResolver.java:94) ~[el-api.jar:3.0.FR]
	... 384 more

      The MongoDB delete command contains the document IDs of all the documents, which exceed a maximum size.

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 4 days, 3 hours, 30 minutes
                4d 3h 30m