Sorry; I thought you might have the orange cloud enabled as you mentioned that you had set SSL in Cloudflare to “Flexible”, this setting only applies for requests proxied through CloudFlare (Aka orange icons). If you don’t have an orange icon next to your records, then you’re not using Cloudflare as anything else then a DNS provider.
I use cloudflare myself, and can’t exactly say that I have any issues, tho I have reported before that Netlify feels quite slow, quite often. I’m not sure if it’s Netlify in itself, Cloudflare or similar, but netlify’s speeds seems to vary drastically.
I’ve read, multiple times, that CNAME usage is not recommended for custom domains, and Netlify recommends that you point the domain directly to Netlify using their nameservers; if this actually changes anything, I couldn’t say, I havn’t tried. I use most of my domains for multiple things, and Netlify’s DNS is a bit limited compared to Cloudflare, to put it kindly 
Tho, they recommend it, so might be worth a shot?
It’s hard to say exactly what the problem is, I experience it myself, and it seems to vary quite a bit. I get an instinct feeling that the more I update my website, the more I build, the slower the website feels, maybe Netlify is slow at sharing the site across the CDN network? I wouldn’t know.
As you can see, then your TTFB is quite slow (Time to first byte), which is represented by green. Blue indicates download time, where most of it is quite fast, except your semi-large jpeg image. Most of the loading time is actually the browser waiting for a response from Netlify.
This below, is an example from one of my sites hosted on Netlify, and I spent quite a lot of time optimizing my code.
As you can see, then my TTFB is quite high as well, tho, not for every request. On a general notice, I feel that Netlify is slow; but it’s free, so I can’t complain. The day I pay for Netlify I would expect a much higher level of response time and speeds. Which is another confusing topic, as some staff member, perhaps a veteran, mentioned that there is no difference in the nodes between free users and paying users, even tho they specifically advertise “High end speeds” as one of the perks for buying a subscription. So not sure what to believe here.
I’ve chunked it all down to “it’s free”, and because it’s free, and because I wouldn’t accept these response times for anything I take seriously, then I’ve spent a minimal amount of time, next to none, trying to figure out what it is or what’s causing it. It’s literally not my problem, but Netlify’s
Sorry; not really sure how this suddenly became an essay of absolutely nothing useful at all 
