-
Type:
Improvement
-
Status: Open
-
Priority:
Minor
-
Resolution: Unresolved
-
Affects Version/s: NoFixVersionApplicable
-
Fix Version/s: Not yet scheduled version
-
Component/s: Local watcher
-
Epic Link:
Current Behavior
When deleting a folder containing a lot of subfolder and files, the process will iterate over each and every subfolder and file.
This is not efficient as Drive only needs to send the top-level folder deletion to the server. All other events will be ignored. But even if they will be ignored, there will be database access and several actions that will just consume resources for nothing.
Here is a sample of logs (deleted the folder named "1000":
... 2021-02-25 15:43:55 63227 123145564352512 DEBUG nxdrive.engine.watcher.local_watcher Queueing watchdog: <FileDeletedEvent: src_path='/Users/tiger-222/Nuxeo Drive/1000/674.txt'> ... 2021-02-25 15:43:55 63227 123145564352512 DEBUG nxdrive.engine.watcher.local_watcher Queueing watchdog: <FileDeletedEvent: src_path='/Users/tiger-222/Nuxeo Drive/1000/900.txt'> 2021-02-25 15:43:55 63227 123145564352512 DEBUG nxdrive.engine.watcher.local_watcher Queueing watchdog: <DirDeletedEvent: src_path='/Users/tiger-222/Nuxeo Drive/1000'>
2021-02-25 15:43:56 63227 123145462005760 INFO nxdrive.engine.watcher.local_watcher Handling watchdog event [deleted] on '/Users/tiger-222/Nuxeo Drive/1000/674.txt'
2021-02-25 15:43:56 63227 123145462005760 DEBUG nxdrive.engine.watcher.local_watcher Watchdog event <FileDeletedEvent: src_path='/Users/tiger-222/Nuxeo Drive/1000/674.txt'> on known pair <DocPair[643] local_path=PosixPath('NXDRIVE-2479/1000/674.txt')...>
2021-02-25 15:43:56 63227 123145462005760 DEBUG nxdrive.engine.watcher.local_watcher Deleted event on inexistent file: PosixPath('/Users/tiger-222/Nuxeo Drive/1000/674.txt')
...
2021-02-25 15:43:59 63227 123145462005760 INFO nxdrive.engine.watcher.local_watcher Handling watchdog event [deleted] on '/Users/tiger-222/Nuxeo Drive/1000'
2021-02-25 15:43:59 63227 123145462005760 DEBUG nxdrive.engine.watcher.local_watcher Watchdog event <DirDeletedEvent: src_path='/Users/tiger-222/Nuxeo Drive/1000'> on known pair <DocPair[3] local_path=PosixPath('NXDRIVE-2479/1000')...>
2021-02-25 15:43:59 63227 123145462005760 DEBUG nxdrive.engine.watcher.local_watcher Deleting file: PosixPath('/Users/tiger-222/Nuxeo Drive/1000')
The folder contained 1,000 files. There were then 999 useless events.
New Behavior
Making use of the local events queue (NXDRIVE-2540), such events could be ingested and one final FS event should be put into the LocalWatcher queue: the top-level folder.
- depends on
-
NXDRIVE-2540 Introduce the local events queue
-
- Open
-