Blocker All LTS jobs are failing at the "Run runtime unit tests" stage with:
[2023-09-12T09:18:07.333Z] [ERROR] Tests run: 10, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 9.294 s <<< FAILURE! - in org.nuxeo.runtime.stream.tests.TestStreamService [2023-09-12T09:18:07.333Z] [ERROR] testStreamMetrics(org.nuxeo.runtime.stream.tests.TestStreamService) Time elapsed: 0.013 s <<< ERROR! [2023-09-12T09:18:07.333Z] java.lang.IllegalArgumentException: Invalid namespace: '1694510264919' [2023-09-12T09:18:07.333Z] at org.nuxeo.lib.stream.log.Name.checkNameSpace(Name.java:135) [2023-09-12T09:18:07.333Z] at org.nuxeo.lib.stream.log.Name.<init>(Name.java:56) [2023-09-12T09:18:07.333Z] at org.nuxeo.lib.stream.log.Name.ofId(Name.java:89) [2023-09-12T09:18:07.333Z] at org.nuxeo.lib.stream.log.NameResolver.getName(NameResolver.java:43) [2023-09-12T09:18:07.333Z] at org.nuxeo.lib.stream.log.kafka.KafkaLogManager.listAllNames(KafkaLogManager.java:217) [2023-09-12T09:18:07.333Z] at org.nuxeo.lib.stream.log.UnifiedLogManager.listAllNames(UnifiedLogManager.java:185) [2023-09-12T09:18:07.333Z] at org.nuxeo.runtime.stream.StreamMetricsComputation.getStreams(StreamMetricsComputation.java:135) [2023-09-12T09:18:07.333Z] at org.nuxeo.runtime.stream.StreamMetricsComputation.getGroups(StreamMetricsComputation.java:151) [2023-09-12T09:18:07.333Z] at org.nuxeo.runtime.stream.StreamMetricsComputation.registerMetrics(StreamMetricsComputation.java:103) [2023-09-12T09:18:07.333Z] at org.nuxeo.runtime.stream.StreamMetricsComputation.refreshMetricsIfNeeded(StreamMetricsComputation.java:128) [2023-09-12T09:18:07.333Z] at org.nuxeo.runtime.stream.StreamMetricsComputation.processTimer(StreamMetricsComputation.java:113) [2023-09-12T09:18:07.333Z] at org.nuxeo.runtime.stream.tests.TestStreamService.testStreamMetrics(TestStreamService.java:239) [2023-09-12T09:18:07.333Z] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Apparently linked to a recent Kubernetes node upgrade from 1.25.10-gke.2700 to 1.25.11-gke.1700:
gcloud container operations list NAME TYPE LOCATION TARGET STATUS_MESSAGE STATUS START_TIME END_TIME operation-1694429351152-6435c8b1-d992-469d-b910-4b7b4eb938b2 UPGRADE_MASTER us-east1-b jx-prod DONE 2023-09-11T10:49:11.152700162Z 2023-09-11T10:54:25.731042109Z operation-1694431787656-05ac67e6-dc1c-4cff-a047-9f918bc350a4 UPGRADE_NODES us-east1-b pool-platform-build DONE 2023-09-11T11:29:47.656111983Z 2023-09-11T21:33:46.648496544Z operation-1694470468351-0ade8c26-4bac-4e71-86d7-e5232aa843e6 UPGRADE_NODES us-east1-b pool-platform-benchmark DONE 2023-09-11T22:14:28.351785377Z 2023-09-11T22:14:51.212622479Z operation-1694471848400-334a0c55-00ac-4d6e-b6f3-7775f26f3f0e UPGRADE_NODES us-east1-b pool-default DONE 2023-09-11T22:37:28.400131564Z 2023-09-11T22:55:03.851439007Z
We can see in the Kafka logs that some low level test topics in nuxeo-stream are not deleted, they have an invalid format for the runtime.
- is related to
-
NXP-32010 Reduce load on Kafka during unit tests
-
- Resolved
-
