-
Type:
Improvement
-
Status: Resolved
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: None
-
Component/s: Automation, Bulk
-
Tags:
-
Team:PLATFORM
-
Sprint:nxplatform #32, nxplatform #33, nxplatform #34, nxplatform #36, nxplatform #35
-
Story Points:0
Today to run a bulk processing the UI uses the BulkRunAction endpoint and invokes the bulk action 'automation' (AutomationBulkAction) with the target operationId that will be applied to all documents.
To be able to run a bulk processing for the "Select All" feature some adaptation are required:
Some document ids can be excluded explicitly
A new excludeDocs parameter has been added to BulkRunAction taking a list of document ids to exclude, the NXQL query used in the bulk command is rewritten accordingly.
Bulk commands for the "Select All" uses a dedicated processor (stream and thread pool)
Instead of calling the 'automation' bulk action, the UI for the 'Select All' purpose needs to call 'automationUi',
it works the same as automation but on a dedicated processor and it supports the query limitation described below.
Bulk commands can be limited depending on the target operationId invoked
The BulkRunAction endpoint has a queryLimit parameter that can be set explicitly.
When this parameter is unspecified the default queryLimit for the automationUi action is taken from the configuration service.
The property name convention is org.nuxeo.ecm.automation.ui.bulk.queryLimit.<<operationId>>
For instance, to limit the number of document processed when invoking the 'Document.Update' through automationUi you have to contribute:
<extension target="org.nuxeo.runtime.ConfigurationService" point="configuration"> <property name="org.nuxeo.ecm.automation.ui.bulk.queryLimit.Document.Update">10000</property> </extension>
Example with curl
Warning this is going to change all dc:description do not run this in any PRODUCTION site:
Run a `Document.Udpate" on all the documents except 2:
curl -s -X POST "http://localhost:8080/nuxeo/site/automation/Bulk.RunAction" -u Administrator:Administrator -H 'Content-Type: application/json' -d '{ "context": {}, "params": { "action": "automationUi", "query": "SELECT * FROM Document", "excludeDocs": ["9cea2a58-c719-4989-8409-c038da5bc8d3", "1f381e2e-f21f-473b-8fdb-9915c7502f60"], "parameters": "{\"operationId\":\"Document.Update\",\"parameters\":{\"properties\":\"dc:description=Update from bulk\"}}" } } response: { "entity-type": "bulkStatus", "value": { "entity-type": "bulkStatus", "commandId": "204092f9-6d3e-49a3-b699-c01e47f16749", "state": "SCHEDULED", "processed": 0, "error": false, "errorCount": 0, "total": 0, "action": "automationUi", "username": "Administrator", "submitted": "2021-05-07T08:08:14.167Z", "scrollStart": null, "scrollEnd": null, "processingStart": null, "processingEnd": null, "completed": null, "processingMillis": 0 } }
Looking at the command executed we see that the NXQL has been rewritten to exclude some docs:
./bin/stream.sh cat -k -l bulk/command --codec avro -schema-store /var/lib/nuxeo/avro | offset | watermark | flag | key | length | data | | --- | --- | --- | --- | ---: | --- | ... |bulk-command-00:+13|2021-05-07 08:08:14.170:0|[DEFAULT]|204092f9-6d3e-49a3-b699-c01e47f16749|315|{"id": "204092f9-6d3e-49a3-b699-c01e47f16749", "action": "automationUi", "query": "SELECT * FROM Document WHERE ecm:uuid NOT IN ('9cea2a58-c719-4989-8409-c038da5bc8d3','1f381e2e-f21f-473b-8fdb-9915c7502f60')", "queryLimit": null, "username": "Administrator", "repository": "default", "bucketSize": 100, "batchSize": 10, "scroller": null, "genericScroller": false, "externalScroller": false, "params": "{\"operationId\":\"Document.Update\",\"parameters\":{\"properties\":\"dc:description=Update from bulk\"}}"}|
