-
Type: New Feature
-
Status: Resolved
-
Priority: Minor
-
Resolution: Fixed
-
Affects Version/s: 10.10
-
Fix Version/s: 10.10-HF23, 11.1, 2021.0
-
Component/s: Monitoring, Streams
-
Release Notes Summary:Nuxeo Stream latency metrics is exposed to Datadog.
-
Release Notes Description:
-
Team:PLATFORM
-
Sprint:nxplatform 11.1.27, nxplatform 11.1.28
-
Story Points:5
Since NXP-26248 stream.sh monitor can expose lag and latency metrics to Graphite.
We want the same feature for Datadog.
Datadog is unable to work properly with wildcard, producing metrics per stream, consumer and partition is very hard to exploit in Datadaog:
server.<hostname>.nuxeo.stream.<stream_name>.<consumer_group>.<partition>.latency
Instead, we need to use simple fewer metrics like:
- nuxeo.streams.lag the lag of the consumer for the stream, in records.
- nuxeo.streams.latency the latency of the consumer for the stream in microsecond.
- nuxeo.streams.pos the last checkpointed position of the consumer in the stream, in record.
- nuxeo.streams.end the end offset of a stream, in record.
And use Datadog tags for the stream, consumer and partition dimensions:
- stream:name the name of the stream
- consumer:group the consumer group name
- partition:partition the partition number like 00, 10 or {{all for aggregated metric for the stream
Also by default we don't need to have metrics per partition, the all aggregation should be good enough and will reduce the number of metrics to 4 per stream consumer.