The initial DBS design does not include any caching and actually at least for standard use cases with MongoDB this does not create any performance issues.
However, recent work have shown that this "no cache" approach has some limitations :
- with the Marklogic backend having cache is critical to be able to get decent performances
- with a big sharded MongoDB cluster, some queries become the bottleneck during the import process
Currently, we have tested both Marklogic and sharded MongoDB with ad-hoc cache :
- directly inside the storage adapter: we have different code for MongoDB and Marklogic
- we have no invalidation management
The goal is to have one shared cache implementation, at DBS level.
On the contrary of VCS where the caching is a built-in concept, the DBS cache should be configurable :
- define cache size
- enable or disable cache
The underlying idea is that since in most MongoDB use cases the cache is not really required, we may want to disable it.
We need to implement an invalidation system based on Redis: we should be able to reuse some of the code (or at least principles).