-
Type:
Bug
-
Status: Open
-
Priority:
Minor
-
Resolution: Unresolved
-
Affects Version/s: None
-
Component/s: Streams
-
Tags:
-
Sprint:nxplatform next
-
Story Points:2
The kv store overflow filter to store big record (> 1MB) is not working properly when using mongodb because of deployment order (stream service being started before mongo) resulting in NPE on start
024-10-11T12:52:27,001 ERROR [RegistrationInfoImpl] Component service:org.nuxeo.stream.defaultConfig notification of application started failed: Cannot invoke "com.mongodb.client.MongoClient.getDatabase(String)" because "mongoClient" is null java.lang.NullPointerException: Cannot invoke "com.mongodb.client.MongoClient.getDatabase(String)" because "mongoClient" is null at org.nuxeo.runtime.mongodb.MongoDBConnectionHelper.getDatabase(MongoDBConnectionHelper.java:256) ~[nuxeo-runtime-mongodb-2023.17.6.jar:?] at org.nuxeo.runtime.mongodb.MongoDBComponent.getDatabase(MongoDBComponent.java:111) ~[nuxeo-runtime-mongodb-2023.17.6.jar:?] at org.nuxeo.ecm.core.mongodb.kv.MongoDBKeyValueStore.initialize(MongoDBKeyValueStore.java:107) ~[nuxeo-core-mongodb-2023.17.6.jar:?] at org.nuxeo.runtime.kv.KeyValueServiceImpl.getKeyValueStore(KeyValueServiceImpl.java:73) ~[nuxeo-runtime-kv-2023.17.6.jar:?] at org.nuxeo.ecm.core.work.KeyValueStoreOverflowRecordFilter.getKeyValueStore(KeyValueStoreOverflowRecordFilter.java:45) ~[nuxeo-core-event-2023.17.6.jar:?] at org.nuxeo.ecm.core.work.KeyValueStoreOverflowRecordFilter.init(KeyValueStoreOverflowRecordFilter.java:41) ~[nuxeo-core-event-2023.17.6.jar:?] at org.nuxeo.ecm.core.work.StreamWorkManager.getRecordFilter(StreamWorkManager.java:268) ~[nuxeo-core-event-2023.17.6.jar:?] at org.nuxeo.ecm.core.work.StreamWorkManager.initTopology(StreamWorkManager.java:386) ~[nuxeo-core-event-2023.17.6.jar:?] at org.nuxeo.ecm.core.work.StreamWorkManager.init(StreamWorkManager.java:306) ~[nuxeo-core-event-2023.17.6.jar:?] at org.nuxeo.ecm.core.work.WorkManagerImpl.start(WorkManagerImpl.java:380) ~[nuxeo-core-event-2023.17.6.jar:?] at org.nuxeo.ecm.core.work.StreamWorkManager.start(StreamWorkManager.java:249) ~[nuxeo-core-event-2023.17.6.jar:?]
The KV store filter is also limited by the kv value size, which is 16MB when using MongoDB.
Even if the kv store filter works on VCS backend, It is much better to rely on the transient store overflow filter and to deprecate this filter.
This means using:
nuxeo.stream.work.computation.filter.class=org.nuxeo.ecm.core.transientstore.computation.TransientStoreOverflowRecordFilter
instead of
nuxeo.stream.work.computation.filter.class=org.nuxeo.ecm.core.work.KeyValueStoreOverflowRecordFilter
