When running some kind of action we don't want that they run in a different order.
For instance if we trash a folder then untrash it, the result depends on the order the commands are executed:
- trash then untrash
- untrash then trash
- trash and untrash at the same time
For other actions we want concurrent execution for performance reason, there is no interest to wait for an export before running a second one.
To get ordered command execution it requires to remove the concurrency, an action can simply define an input stream of one partition but this will not prevent the scrolling to interleave buckets (commands will overlap in this case).
To prevent this, commands must be sharded accordingly so the scroller is not running concurrently for such action.
- adding a sequentialCommands flag on the action descriptor
- when submitting the command if this flag is set, the shard key will be the action id instead of a random one, this way scrolling will be sequential for this action
- the action processor input stream must have only one partition to prevent commands overlap. Event if buckets are not interleaved on partitions, it is possible to have computations processing different commands at the same time.
Actions that support concurrent command:
- setProperties ?
Actions that need to have sequential command