-
Type: Improvement
-
Status: Resolved
-
Priority: Minor
-
Resolution: Fixed
-
Affects Version/s: 10.10
-
Fix Version/s: 10.10-HF21, 11.1, 2021.0
-
Component/s: Streams
-
Release Notes Summary:Elasticsearch scroller is usable with the Bulk Service.
-
Release Notes Description:
-
Backlog priority:700
-
Sprint:nxplatform 11.1.22, nxplatform 11.1.23, nxplatform 11.1.25, nxplatform 11.1.24, nxplatform 11.1.26
-
Story Points:1
The Bulk Service scroller responsible to materialize the document set is using a repository query (with the scroll API to manage long-running query).
The goal of the Bulk Service being to process massively documents it makes sense to query the repository using the backend which is the source of truth.
For some non-critical action like CSV export or AI export, we want to be able to use Elasticsearch to materialize the document set because:
- the query may come from an Elasticsearch Page Provider and can only be performed by Elasticsearch when using ES hints or aggregation filters.
- the query may uses a full-text field that has been disabled at the repository level (nuxeo.vcs.fulltext.search.disabled=true) for performance reason.
We should provide an option at the action and/or command level to choose the type of scroller to use.
—
To set the default scroller for a Bulk Action use the defaultScroller option:
<action name="csvExport" bucketSize="100" batchSize="50" httpEnabled="true" defaultScroller="elastic" validationClass="org.nuxeo.ecm.platform.csv.export.validation.CSVExportValidation"/>
To choose the scroller at Bulk Command level when using a Page Provider
curl -X POST "http://localhost:8080/nuxeo/api/v1/search/pp/default_search/bulk/csvExport?scroll=elastic" -u Administrator:Administrator
- depends on
-
NXP-28334 Create a scroll service
- Resolved
- is related to
-
NXP-28089 Web UI dev mode: async bulk status poll hangs forever
- Open
-
NXP-30570 Bulk command scroller should be aligned on the underlying page provider
- Resolved
-
NXP-28356 Enable to register a Processor without running it
- Resolved
- Is referenced in