Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-31278

Fix random Postgres Concurrent update exception on DocumentRoutingServiceImpl#doCleanupDoneAndCanceledRouteInstances

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: 2021.x, 2023.x
    • Component/s: Workflow
    • Tags:
    • Sprint:
      nxplatform #71
    • Story Points:
      3

      Description

      The method DocumentRoutingServiceImpl.doCleanupDoneAndCanceledRouteInstances may produces random Concurrent update exception on postgres.

      Seen in https://jenkins.platform.dev.nuxeo.com/blue/organizations/jenkins/nuxeo%2Flts%2Fnuxeo/detail/PR-511/1/tests:

      Error
      Failed to remove document 25ebdf85-da0c-4714-be17-51902f4b9598, Concurrent update
      Stacktrace
      org.nuxeo.ecm.core.api.ConcurrentUpdateException: Failed to remove document 25ebdf85-da0c-4714-be17-51902f4b9598, Concurrent update
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCConnection.checkConcurrentUpdate(JDBCConnection.java:172)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCRowMapper.deleteRowsDirect(JDBCRowMapper.java:821)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCRowMapper.remove(JDBCRowMapper.java:1259)
      	at org.nuxeo.ecm.core.storage.sql.SoftRefCachingRowMapper.remove(SoftRefCachingRowMapper.java:453)
      	at org.nuxeo.ecm.core.storage.sql.PersistenceContext.removeNode(PersistenceContext.java:1001)
      	at org.nuxeo.ecm.core.storage.sql.SessionImpl.removeNode(SessionImpl.java:919)
      	at org.nuxeo.ecm.core.storage.sql.coremodel.SQLSession.remove(SQLSession.java:598)
      	at org.nuxeo.ecm.core.storage.sql.coremodel.SQLDocumentLive.remove(SQLDocumentLive.java:165)
      	at org.nuxeo.ecm.core.storage.sql.coremodel.SQLDocumentLive.remove(SQLDocumentLive.java:171)
      	at org.nuxeo.ecm.core.api.AbstractSession.removeNotifyOneDoc(AbstractSession.java:1535)
      	at org.nuxeo.ecm.core.api.AbstractSession.removeDocument(AbstractSession.java:1507)
      	at org.nuxeo.ecm.core.api.AbstractSession.removeDocument(AbstractSession.java:1497)
      	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
      	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
      	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
      	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
      	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
      	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
      	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
      	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
      	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
      	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
      	at org.nuxeo.ecm.platform.routing.core.impl.DocumentRoutingServiceImpl$WfCleaner.run(DocumentRoutingServiceImpl.java:962)
      	at org.nuxeo.ecm.core.api.UnrestrictedSessionRunner.runUnrestricted(UnrestrictedSessionRunner.java:123)
      	at org.nuxeo.ecm.platform.routing.core.impl.DocumentRoutingServiceImpl.doCleanupDoneAndCanceledRouteInstances(DocumentRoutingServiceImpl.java:1274)
      	at org.nuxeo.ecm.platform.routing.core.impl.DocumentRoutingServiceImpl.cleanupDoneAndCanceledRouteInstances(DocumentRoutingServiceImpl.java:1268)
      	at org.nuxeo.ecm.platform.routing.test.GraphRouteTest.testRouteWithTasks(GraphRouteTest.java:957)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$BeforeMethodRunStatement.evaluate(FeaturesRunner.java:457)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$BeforeSetupStatement.evaluate(FeaturesRunner.java:471)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner.evaluateRunnable(FeaturesRunner.java:328)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$AfterMethodRunStatement.evaluate(FeaturesRunner.java:493)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner.evaluateRunnable(FeaturesRunner.java:328)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$AfterTeardownStatement.evaluate(FeaturesRunner.java:506)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$RulesFactory$1.evaluate(FeaturesRunner.java:576)
      	at org.nuxeo.runtime.test.runner.RuntimeDeployment$DeploymentStatement.evaluate(RuntimeDeployment.java:303)
      	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
      	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
      	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
      	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$AfterClassStatement.evaluate(FeaturesRunner.java:404)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$RulesFactory$1.evaluate(FeaturesRunner.java:576)
      	at org.nuxeo.runtime.test.runner.FeaturesRunner$BeforeClassStatement.evaluate(FeaturesRunner.java:387)
      	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
      	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
      	at org.junit.runners.Suite.runChild(Suite.java:128)
      	at org.junit.runners.Suite.runChild(Suite.java:27)
      	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
      	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
      	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
      	at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
      	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
      	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
      	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
      	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
      	at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
      	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
      	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
      Caused by: org.postgresql.util.PSQLException: ERROR: deadlock detected
        Detail: Process 42283 waits for ShareLock on transaction 355336; blocked by process 42263.
      Process 42263 waits for ShareLock on transaction 355335; blocked by process 42283.
        Hint: See server log for query details.
        Where: while deleting tuple (5,41) in relation "fulltext"
      SQL statement "DELETE FROM ONLY "public"."fulltext" WHERE $1::pg_catalog.text OPERATOR(pg_catalog.=) "id"::pg_catalog.text"
      	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675)
      	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2365)
      	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:355)
      	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:490)
      	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:408)
      	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166)
      	at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:134)
      	at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136)
      	at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCRowMapper.deleteRowsDirect(JDBCRowMapper.java:817)
      	... 78 more
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              grenard Guillaume Renard
              Participants:
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: