Thanks for your help, but I've managed to find the problem for the very slow loading time. So no need for AJAX anymore.
The first thing I spotted was that it only loading extremely slow in Chrome and not Safari or Firefox. I then did a couple of Lighthouse page speed analysis, and removed unused stylesheets etc...
It was the damn "wp-block-library" stylesheet. I've dequeued this and problem solved!
