What an awesome solutions are presented up here, thank you all so much. I was playing a bit more with this and stumbled on a few "bugs" (dont know how to call them differently).
1. When the container has a padding the cursor image is automatically off center (the amount of pixels of the padding)
2. When you re-use the container (right after) again, it doubles that padding space
You can see the two bugs in the codepen down below.
I could resolve the padding issue by adding a -40 to the 'setY(e.pageY -40);' but I'm wondering if there is a cleaner solution, especially keeping point 2 in mind. https://codepen.io/pixelarchitect/pen/ZEyzQWm