[NXP-29631] Fix nuxeoctl start hang - Nuxeo Issue Tracker

XML

Word

Printable

Details

Type: Bug
Status: Resolved
Priority: Blocker
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 10.10-HF32, 11.3, 2021.0
Component/s: Elasticsearch, nuxeoctl start/stop/admin, Runtime

Tags:
Team:
PLATFORM
Sprint:
nxplatform #17
Story Points:
1

Description

We've detected on nuxeo-bench jobs that nuxeoctl start hang after having started the server.

After a quick analysis, nothing obvious was seen (logs, jstack...).

The culprit was the Elasticsearch client leaving a thread pool behind, even if these threads don't appear as blocked on the thread dump.

You can find below a representative stack for this remaining pool:

"I/O dispatcher 8" #25 prio=5 os_prio=31 cpu=3.53ms elapsed=54.38s tid=0x00007f8d0f347800 nid=0x7303 runnable  [0x000070001039e000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueue.poll(java.base@11.0.6/Native Method)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(java.base@11.0.6/KQueueSelectorImpl.java:122)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@11.0.6/SelectorImpl.java:124)
	- locked <0x000000070d498b70> (a sun.nio.ch.Util$2)
	- locked <0x000000070d498a18> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(java.base@11.0.6/SelectorImpl.java:136)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:255)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
	at java.lang.Thread.run(java.base@11.0.6/Thread.java:834)