I figured it out and the answer is very simple.
If your website is in maintenance mode, it will likely block everything but your website's admins and Paypal isn't an admin user for your site ;) Try it yourself:
Type in this url while not logged in on the site (replace example.com for your own url...)
http://example.com/?wc-api=WC_Gateway_Paypal
If it's blocked, that's the problem ;)
You can also activate the logs in the paypal payment gateway params just in case.