CAS+Anonymous: requesting a protected binary using an authorized user leads to CAS logout

When logged in as an authorized user in a CAS+Anonymous configuration, you might be thrown out of CAS when requesting a binary.

How to reproduce:
Use a CAS+Anonymous configuration (+demo DB content)
Anonymous, members and administrators are authorized at Domain level, but forbidden underneath in this case (only members and admnistrators are authorized on Workspaces).
Login as a "members" user such as "bob" from http://IP:PORT/nuxeo
Goto: http://IP:PORT/nuxeo/nxpath/default/default-domain/workspaces/Nuxeo%20Marketing%20Content/Brochures/Nuxeo%20Platform%20%26%20AWS%20bro@view_documents?tabIds=MAIN_TABS%3Adocuments%2C%3A&conversationId=0NXMAIN => all fine
Click on the PDF to download and open it => You are logged out from CAS!

Requesting a binary should not lead to CAS logout out of the box.