I tested window.dragTo on the codepen, but it does not work anymore, the content is simply not scrolled.
I've played a bit with Draggable and found out how to solve my problem :
in onPress :
startPointerY = self.pointerY = e.pageY - window.scrollY;
in onMove
setPointerPosition(e.pageX, e.pageY - window.scrollY);
This way, even if window.scrollY is modified during drag, it keeps the correct start pointer.
Bertrand Larrieu's post in Draggable strange behaviour was marked as the answer