Query Parameters in 200 rewrites through _redirects not working

We’re trying to use a rather complex setup for scripts included on the page, using Gatsby / Partytown:

The redirects part mentioned there as “Hosting on other providers requires support for Gatsby’s createRedirect action to rewrite requests from /__third-party-proxy?url=${YOUR_URL} to YOUR_URL with a 200 status code. You may need to check with your hosting provider to see if this is supported.” is handled by the following Gatsby plugin:

It looks like that is all working as expected… we get a _redirects file created, with all the entries as intended. Those _redirects also all get read correctly when deploying to Netlify (they all get processed). But it’s not working for cases where there’s a Query Parameter as part of the URL targeted by the rewrite rule.

Any idea on where this might go wrong? Following the docs, for rewrites (200), it should just pass these parameters on?

Some more details: entries without Query Parameters seem to work just fine, like this one:
“/__third-party-proxy url=https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js 200”

It works: https://www.kickstartds.com/__third-party-proxy?url=https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js

But when the URL includes a Query Parameter, it seems to fail, for example:
“/__third-party-proxy url=https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV 200”

This doesn’t work: https://www.kickstartds.com/__third-party-proxy?url=https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV

The copied lines come from _headers generated by the Netlify plugin for Gatsby.

Hi @julrich sorry but can you share your site + repo?

Sure, site is https://www.kickstartDS.com / kickstartds-website.netlify.app.
Repository is the following one: https://github.com/kickstartDS/website (not sure how immediately useful that is, though, because you have to be quite deep into Gatsby I guess)

Sorry, just noticed that repository is currently Private anyways… what would be the part of interest there? If this helps: We don’t have a netlify.toml, and all _headers and _redirects are generated by the Gatsby plugin, based on configuration done through Gatsby config mechanism.

I don’t think there’s anything else related to Netlify at play here. See the following screenshot for a deployment of this, where all those redirects seem to be processed successfully:

Full file contents for _headers file:

Full file contents for _redirects file:

(sorry I had to split those into multiple replies, but I’m only allowed one screenshot per reply as a newcomer)

I’m not sure how the plugin works for query parameters syntax. However, I do not think your goal is achievable the way you need. The correct syntax for your requirement would look like:

/___third-party-proxy url=:url :url 200!

But this would be invalid syntax and is not supported.

I believe you can try using:

/___third-party-proxy url=:url https://:url 200!

and then, you won’t have to add any redirect rules to Gatsby, just this one rule should be able to handle it all. But, you need to make sure not to start your URLs with http:// prefix, as that’s something that would be added by us. But again, I’m not confident in this suggestion.

1 Like

So it’s not possible to have redirects with Query Parameters, otherwise? I could easily opt out of that part of the plugin, I’m just not sure how the correct entry (with a Query Parameter) would look like in _redirects.

The entries generated without a Query Param work just as expected, like this one:

/__third-party-proxy url=https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js  https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js  200

There’s no way (even hard-coded) to add the Query Param to the destination? It wouldn’t have to be a splat, I’d actually prefer to have the URLs added to the proxy be super explicit… we don’t need an “open proxy” that can work with arbitrary URLs.

Also experimented with URL encoding (because of the second ? in /__third-party-proxy url=https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV 200), and a bunch of other stuff. But seems the moment the destination path includes a Query Parameter, it just stops working entirely.

Also tried just adding the rewrite as:

/__third-party-proxy url=https://www.googletagmanager.com/gtag/js  https://www.googletagmanager.com/gtag/js  200

with the hope that the Query Parameter (like id) would just be passed through as per the docs, but that also seems not to be the case:

While our service automatically passes on all query string parameters to destination paths for redirects with 200, 301, and 302 HTTP status, you can also choose to define a redirect path based on a specific parameter or combination of parameters.

Sorry, I’m still a little confused as to what it is that you’re looking for.

I deployed this redirect:

/proxy url=https://www.google.com/ https://www.google.com/?q=foo 200!

here:

and I can see it being proxied to Google with the query params in the URL. What am I missing?

1 Like

I think just the Query Param on the URL, but that could be added like this, I guess?

/proxy url=https://www.googletagmanager.com/gtag/js  id=G-GTP95B7SBV https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV 200!

I’ll test that. If I just opt out of the _redirects behavior of the addon, and add those rules myself I could see this work.

The goal is to just proxy those specific tracking scripts (with their Query Params, which often mark the account or property being tracked, for example). And as those calls can ultimately also come from third-party scripts, we need to be exact in the rewrites used for this to work. Does that make sense?

Okay, some testing later:

If I do the redirects (manually) like this:

/__third-party-proxy url=https://www.googletagmanager.com/gtag/js id=G-GTP95B7SBV  https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV  200!
/__third-party-proxy url=https://www.google-analytics.com/analytics.js  https://www.google-analytics.com/analytics.js  200!
/__third-party-proxy url=https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js  https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js  200!
/__third-party-proxy url=https://static.hotjar.com/c/hotjar-2491743.js sv=6  https://static.hotjar.com/c/hotjar-2491743.js?sv=6  200!

(this is also the version online currently on https://www.kickstartDS.com)

I almost get there. I think the main problem with all of this is the second question mark that ends up in the resulting call. Doing it like above avoids this by “replacing” the ? with &. Proxy answers successfully for that. Unfortunately the real links being called are generated by convention, through Partytown.

This links works (with the ? replaced by &, and the above rules in _redirects being active):
https://www.kickstartds.com/__third-party-proxy?url=https://www.googletagmanager.com/gtag/js&id=G-GTP95B7SBV

The real GTM call being made looks like this:

https://www.kickstartds.com/__third-party-proxy?url=https%3A%2F%2Fwww.googletagmanager.com%2Fgtag%2Fjs%3Fid%3DG-GTP95B7SBV

So Partytown already URL encodes those values. This lead to me testing just adding the URL encoded version for url in _redirects, too:

/__third-party-proxy url=https%3A%2F%2Fwww.googletagmanager.com%2Fgtag%2Fjs%3Fid%3DG-GTP95B7SBV  https://www.googletagmanager.com/gtag/js?id=G-GTP95B7SBV  200!

Unfortunately no dice, my guess would be there’s some URL decoding happening before the redirect is actually hitting the `redirects comparison?

Any more ideas? :smiley: Grasping at straws here!

At this point, I’d suggest using Netlify Edge Functions for a much more fine-grained configurable options, and debugging experience: Edge Functions API | Netlify Docs. While you can’t use rewrite() method, you can definitely fetch the data and return that as the response.

I see. Was kinda hoping I could avoid that, as it seemed like a straigh-forward enough proposition by those addon docs. But I think I’ll give that a try.

But you don’t see any way to use encoding / decoding to your advantage to get that, right?

Sorry about the delay, I missed the thread.

But to answer your question, I’d think even if we somehow did find a way, it would get very tricky and difficult to maintain. Should something change with the redirects engine, I’m not sure if your use case would be considered or tested. This is why I suggested Edge Functions as a better option.