-
Type: Improvement
-
Status: Open
-
Priority: Minor
-
Resolution: Unresolved
-
Affects Version/s: 10.10-HF25, 11.1-SNAPSHOT
-
Fix Version/s: None
-
Component/s: Bulk, Performance
-
Epic Link:
Context
During the first Bulk Reindex tests on 200M documents we experienced some issues:
- indexing was never finishing
- Kafka storage was growing almost indefinitely (filled 1.5 TB for reindexing 200GB)
The root cause seems to be a transaction timeout during the initial scroll => redo on each of the nodes.
Proposed solution
To fix the issue, we increased the global TX timeout to several hours.
Ideally, it would be better for the BAF to take care of that and explicitly manage transaction timeout and not rely on a server-wide setting to be applied.