Uploaded image for project: 'Nuxeo Enhanced Viewer'
  1. Nuxeo Enhanced Viewer
  2. NEV-544

Fix the IllegalArgumentException thrown by Prometheus CollectorRegistry

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: arender-2.1.0
    • Component/s: ARender

      Description

      When enabling the Datadog metrics with help of micrometer-metrics library, we observe the following stack trace in the broker logs:

      2022-01-20 16:27:40,651 ERROR [http-nio-0.0.0.0-8761-exec-9] [org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[dispatcherServlet]] Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception
      java.lang.IllegalArgumentException:      arender_weather_seconds_count
                      at io.prometheus.client.CollectorRegistry.register(CollectorRegistry.java:54)
                      at io.prometheus.client.Collector.register(Collector.java:139)
                      at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$applyToCollector$16(PrometheusMeterRegistry.java:410)
                      at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1908)
                      at io.micrometer.prometheus.PrometheusMeterRegistry.applyToCollector(PrometheusMeterRegistry.java:406)
                      at io.micrometer.prometheus.PrometheusMeterRegistry.newTimer(PrometheusMeterRegistry.java:200)
                      at io.micrometer.core.instrument.MeterRegistry.lambda$timer$2(MeterRegistry.java:310)
                      at io.micrometer.core.instrument.MeterRegistry.getOrCreateMeter(MeterRegistry.java:614)
                      at io.micrometer.core.instrument.MeterRegistry.registerMeterIfNecessary(MeterRegistry.java:568)
                      at io.micrometer.core.instrument.MeterRegistry.timer(MeterRegistry.java:308)
                      at io.micrometer.core.instrument.Timer$Builder.register(Timer.java:400)
                      at io.micrometer.core.instrument.composite.CompositeTimer.registerNewMeter(CompositeTimer.java:140)
                      at io.micrometer.core.instrument.composite.CompositeTimer.registerNewMeter(CompositeTimer.java:31)
                      at io.micrometer.core.instrument.composite.AbstractCompositeMeter.add(AbstractCompositeMeter.java:66)
                      at java.base/java.lang.Iterable.forEach(Iterable.java:75)
                      at java.base/java.util.Collections$SetFromMap.forEach(Collections.java:5581)
                      at io.micrometer.core.instrument.composite.CompositeMeterRegistry.lambda$new$0(CompositeMeterRegistry.java:65)
                      at io.micrometer.core.instrument.composite.CompositeMeterRegistry.lock(CompositeMeterRegistry.java:184)
                      at io.micrometer.core.instrument.composite.CompositeMeterRegistry.lambda$new$1(CompositeMeterRegistry.java:65)
                      at io.micrometer.core.instrument.MeterRegistry.getOrCreateMeter(MeterRegistry.java:624)
                      at io.micrometer.core.instrument.MeterRegistry.registerMeterIfNecessary(MeterRegistry.java:568)
                      at io.micrometer.core.instrument.MeterRegistry.timer(MeterRegistry.java:308)
                      at io.micrometer.core.instrument.Timer$Builder.register(Timer.java:400)
                      at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.getTimer(WebMvcMetricsFilter.java:185)
                      at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.record(WebMvcMetricsFilter.java:151)
                      at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:125)
                      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
                      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                      at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
                      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
                      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
                      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
                      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
                      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
                      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
                      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
                      at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
                      at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
                      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
                      at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
                      at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
                      at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
                      at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
                      at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
                      at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
                      at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
                      at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                      at java.base/java.lang.Thread.run(Thread.java:829)
      

      This may be happening when the broker weather endpoint is called by Kubernetes.

      After digging the internet, it may be related to this issue: https://github.com/micrometer-metrics/micrometer/issues/2399

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                kleturc Kevin Leturc
                Participants:
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: