Robots.txt to my netlify deployed cloudfront images

Support
1

Hey!

I’m hoping to de-index some of website images on netlify. I’ve added them to a robots.txt and tested with webmasters. All seems to be working, however because my images are hosted on cloudfront, i’m not confident they’re getting de-indexed.

Can someone confirm this?

I also came across this stack overflow post:

is there a way I can add a subdomain for my image assets to they don’t appear as a cloudfront link… and hopefully I can apply a robots.txt through the subdomain?

Cheers,
Connor

2

Came across this in the netlify docs:

NETLIFY_IMAGES_CDN_DOMAIN:** if image compression post processing is enabled, the base URL used for all processed images; for example, d33wubrfki0l68.cloudfront.net.

So it looks like I was compressing all my images… which is good. But if I untick this option, will those images on robots.txt now work? Is there a way to get lossless while maintaining robots.txt?

#

3

Hey @connorforsythco , just to clarify, do you have post-processing enabled right now? You mentioned unticking that option, but since we don’t know your site URL we can’t verify on our end. Please let us know and we can advise further!

4

@amelia I went ahead and unticked it, just around the same time as posting last night. So far the images are still appearing on search results and the CDN links are still valid, such as this one.

Is it possible to still use post-prossessing and point a CNAME of CDN to a subdomain of mine (like how they did in the stack overflow link)? Curious about that.

My site is https://connorforsyth.co/ if you want to check it out.

Some of the things I’m trying to solve is this image appearing in search results. (I don’t think this image preview is opengraph… I’m also digging around on the internet trying to find a solution for that.)

image
image1542×322 61 KB

And things like this appearing in google image search.

7

Hi @connorforsythco,

Sorry about the delay. Actually, this has been escalated internally and it’s waiting for feedback. There’s no way for you to block those assets now, but we’re confirming with the team if we can offer an alternative.

1 Like
9

Hey @hrishikesh . Thanks for the update. Looking forward to hearing from you.

I tried to change the file name of that specific image. Seen in google search. However I think the old version of the image is still available via cloudfront.

Requested indexing again from google to see if it changes anything.

Cheers,
Connor.

10

Hey @connorforsythco,

The above mentioned URL: https://d33wubrfki0l68.cloudfront.net/3464fc459a115f64135f2e0ef63ad0a745a9f716/86014/images/civic-icon.png, should now be removed from the CDN.