Videos disappeared from site but still appearing on temporary site

Hi, @elhansson (and everyone else in this topic). Thanks for sending the x-nf-request-id as it helped me find the issue.

This is a bug and I’ve gotten an issue filed for it.

To summarize, your sites are using an HTTP header called “range” to request only parts of the file and our CDN has started (sometimes - not all the time) sending incorrect data for this header. @elhansson, here is an real test of the file for the x-nf-request-id: 01FXHZMDET4S2MKSYTWBWA5H5S.

I’m sure you know the real URL so please feel free to replace <REDACTED URL> with the real URL to reproduce the issue locally.

First, I checked to see what the checksum was for the last 347620 bytes of data (because the file is 2347620 bytes and I used 2000000- for the range):

$ curl --compressed -s -H "range: bytes=2000000-"  <REDACTED URL> | md5sum
ca46a203923a70f229ab76c704743d33  -

Then I checked to see the checksums for the first and last 347620 bytes by requesting the full file and using head and tail to filter for just the start and end of the file. MD5 checksum are used for both of those as well.

$ curl --compressed -s <REDACTED URL> | head -c 347620 | md5sum
ca46a203923a70f229ab76c704743d33  -
$ curl --compressed -s <REDACTED URL> | tail -c 347620 | md5sum
bda33ecb2c07e86b1445e77609295e59  -

As you can see, this proves the range request asking for the end of the file (the tail) actually got the data for the start of the file (the head). The wrong 347620 bytes is sent hence the errors for the site code expecting the correct data.

Please note, I cannot always reproduce this 100% of the time. For example, I cannot trigger this now for your file, @kaanna. I correctly get the end of the file when I test. I’m almost certain that your site is being affected by the same issue. I just cannot reliably reproduce it for that URL.

Also, this seems to primarily impact files over 1 MB in size. I cannot trigger the behavior with smaller files only with those about 2 MB or larger.

We will update this topic as soon as we have more information (or to say the issue is fixed).

I appreciate everyone letting us know about this and the specific examples (like your x-nf-request-id, @elhansson) were very helpful . Thank you for taking the time to send these examples to us.

2 Likes