Improve the NuxeoDrive.ScrollDescendants contract

Current Behavior

The NuxeoDrive.ScrollDescendants operation contract is to return at most batchSize documents.

This is because the batch size represents the max number of descendant docs IDs to handle (query), then they get adapted to FileSystemItem in Java, so this post-filtering can remove some docs comparing to the max size, typically the docs for which permissions are denied.

Implied Issue

On the Nuxeo Drive side, with NXDRIVE-1636, the current behavior turned out to broke the synchronization of deep structures (see NXDRIVE-1832). At that time, I was not aware of that contract before applying the patch in NXDRIVE-1636.
And at the same time, saving 1 HTTP call (check NXDRIVE-1636 for details) is always a good thing, more importantly now that we are going more and more to the cloud.

To be more explicit, when Drive syncs a deep structure, this is the amount of documents retrieved each time the operation is called (with a batch size of 500):

499  <-- should be 500
499  <-- should be 500
500
500
500
500
500
499  <-- should be 500
500
500
85

Here, the 85 means really there are no more docs.

So with the improvement included in Drive with NXDRIVE-1636, the synchronization is stopped at the first call because 499 is less than 500. Drive understands there is no more documents to sync.

Improvement

The improvement proposed is to modify the NuxeoDrive.ScrollDescendants contract to return exactly batchSize documents.

Antoine > I guess it could be possible to try to return a max number of docs, let's say if I only adapt 499 (the 500th is denied) but there are more available, to scroll again and check if the 501th is OK, include it, etc. to retrieve exactly 500.
Antoine > But this implies a behavior change, so might not be easily backportable, and not trivial.

With that new contract, when Drive will get less documents than the batch size would really telling there is no more document. And so the saved HTTP call in NXDRIVE-1636 will be enabled again.