Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-19335

MSSQL deadlock on delete in reference benchmark

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 8.1
    • Fix Version/s: QualifiedToSchedule
    • Component/s: Core VCS

      Description

      There are plenty of errors on the CRUD simulation on delete.

      https://benchmarks.nuxeo.com/build/362/archive/reports/sim50crud/detail/req_delete-file-7d1e2.html

      deletion is very long and server.log shows transaction timeout and deadlock:
      https://benchmarks.nuxeo.com/build/362/archive/logs/172.30.2.79/

      2016-03-23 22:52:00,715 WARN  [http-bio-0.0.0.0-8080-exec-349] [org.nuxeo.ecm.webengine.app.WebEngineExceptionMapper] Exception in JAX-RS processing
      org.nuxeo.ecm.webengine.WebException: Failed to delete document /default-domain/workspaces/Bench_Gatling/u09/t/g/9/u09tg9h4q5jp
      	at org.nuxeo.ecm.webengine.WebException.wrap(WebException.java:160)
      	at org.nuxeo.ecm.core.rest.DocumentObject.doDelete(DocumentObject.java:121)
      	at org.nuxeo.ecm.restapi.server.jaxrs.JSONDocumentObject.doDelete(JSONDocumentObject.java:107)
      	at sun.reflect.GeneratedMethodAccessor692.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
      	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
      	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
      	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
      	at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
      	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
      	at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
      	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
      	at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
      	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
      	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
      	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
      	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
      	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
      	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
      	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
      	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
      	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
      	at org.nuxeo.ecm.webengine.app.jersey.WebEngineServlet.containerService(WebEngineServlet.java:178)
      	at org.nuxeo.ecm.webengine.app.jersey.WebEngineServlet.service(WebEngineServlet.java:155)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.webengine.jaxrs.session.SessionCleanupFilter.run(SessionCleanupFilter.java:50)
      	at org.nuxeo.ecm.webengine.jaxrs.HttpFilter.doFilter(HttpFilter.java:49)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.webengine.app.WebEngineFilter.doFilter(WebEngineFilter.java:94)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.webengine.jaxrs.context.RequestContextFilter.run(RequestContextFilter.java:48)
      	at org.nuxeo.ecm.webengine.jaxrs.HttpFilter.doFilter(HttpFilter.java:49)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.core.management.jtajca.internal.Log4jWebFilter.doFilter(Log4jWebFilter.java:69)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.ui.web.auth.NuxeoAuthenticationFilter.doFilterInternal(NuxeoAuthenticationFilter.java:586)
      	at org.nuxeo.ecm.platform.ui.web.auth.service.NuxeoAuthFilterChain.doFilter(NuxeoAuthFilterChain.java:51)
      	at org.nuxeo.ecm.platform.ui.web.auth.oauth.NuxeoOAuthFilter.doFilter(NuxeoOAuthFilter.java:122)
      	at org.nuxeo.ecm.platform.ui.web.auth.service.NuxeoAuthFilterChain.doFilter(NuxeoAuthFilterChain.java:49)
      	at org.nuxeo.ecm.platform.ui.web.auth.oauth2.NuxeoOAuth2Filter.doFilter(NuxeoOAuth2Filter.java:83)
      	at org.nuxeo.ecm.platform.ui.web.auth.service.NuxeoAuthFilterChain.doFilter(NuxeoAuthFilterChain.java:49)
      	at org.nuxeo.ecm.platform.ui.web.auth.NuxeoAuthenticationFilter.doFilter(NuxeoAuthenticationFilter.java:399)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.web.common.exceptionhandling.NuxeoExceptionFilter.doFilter(NuxeoExceptionFilter.java:77)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.web.common.encoding.NuxeoEncodingFilter.doFilter(NuxeoEncodingFilter.java:75)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:748)
      	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:486)
      	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:411)
      	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:338)
      	at org.nuxeo.ecm.restapi.server.APIServlet.service(APIServlet.java:48)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.web.common.requestcontroller.filter.NuxeoThreadTrackerFilter.doFilter(NuxeoThreadTrackerFilter.java:43)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.core.management.jtajca.internal.Log4jWebFilter.doFilter(Log4jWebFilter.java:69)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.ui.web.rest.FancyURLFilter.doFilter(FancyURLFilter.java:120)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.web.common.requestcontroller.filter.NuxeoRequestControllerFilter.doFilter(NuxeoRequestControllerFilter.java:142)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.webdav.service.WIRequestFilter.doFilter(WIRequestFilter.java:61)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.web.common.requestcontroller.filter.NuxeoCorsFilter.doFilter(NuxeoCorsFilter.java:51)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.web.common.exceptionhandling.NuxeoExceptionFilter.doFilter(NuxeoExceptionFilter.java:77)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.nuxeo.ecm.platform.web.common.encoding.NuxeoEncodingFilter.doFilter(NuxeoEncodingFilter.java:75)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
      	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
      	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
      	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
      	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
      	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
      	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
      	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
      	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
      	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.nuxeo.ecm.core.api.ConcurrentUpdateException: Failed to remove document 2ab5f929-c393-4cb3-abe1-7d503ec387e4, java.sql.SQLException: Transaction (Process ID 99) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCConnection.checkConcurrentUpdate(JDBCConnection.java:210)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCRowMapper.deleteRowsDirect(JDBCRowMapper.java:865)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCRowMapper.remove(JDBCRowMapper.java:1305)
      	at sun.reflect.GeneratedMethodAccessor693.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCMapperConnector.doInvoke(JDBCMapperConnector.java:42)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCMapperConnector.invoke(JDBCMapperConnector.java:85)
      	at com.sun.proxy.$Proxy64.remove(Unknown Source)
      	at org.nuxeo.ecm.core.storage.sql.SoftRefCachingRowMapper.remove(SoftRefCachingRowMapper.java:436)
      	at org.nuxeo.ecm.core.storage.sql.PersistenceContext.removeNode(PersistenceContext.java:870)
      	at org.nuxeo.ecm.core.storage.sql.SessionImpl.removeNode(SessionImpl.java:1088)
      	at org.nuxeo.ecm.core.storage.sql.ra.ConnectionImpl.removeNode(ConnectionImpl.java:246)
      	at org.nuxeo.ecm.core.storage.sql.coremodel.SQLSession.remove(SQLSession.java:606)
      	at org.nuxeo.ecm.core.storage.sql.coremodel.SQLDocumentLive.remove(SQLDocumentLive.java:164)
      	at org.nuxeo.ecm.core.api.AbstractSession.removeNotifyOneDoc(AbstractSession.java:1333)
      	at org.nuxeo.ecm.core.api.AbstractSession.removeDocument(AbstractSession.java:1303)
      	at org.nuxeo.ecm.core.api.AbstractSession.removeDocument(AbstractSession.java:1293)
      	at org.nuxeo.ecm.core.rest.DocumentObject.doDelete(DocumentObject.java:118)
      	... 108 more
      Caused by: java.sql.SQLException: Transaction (Process ID 99) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
      	at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:372)
      	at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2893)
      	at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2335)
      	at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:638)
      	at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:613)
      	at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:572)
      	at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeUpdate(JtdsPreparedStatement.java:727)
      	at org.tranql.connector.jdbc.PreparedStatementHandle.executeUpdate(PreparedStatementHandle.java:186)
      	at org.nuxeo.ecm.core.storage.sql.jdbc.JDBCRowMapper.deleteRowsDirect(JDBCRowMapper.java:858)
      	... 125 more
      

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: