HSTS mandates https. Browsers have a list of trusted root CAs, you need to serve a certificate signed by one of those to visitors.
Without Cloudflare, this means one of them directly, like Let's Encrypt
With Cloudflare in the middle, you should either use a trusted CA directly like Let's Encrypt, or you can use a Cloudflare Origin Cert, either will work