Don't reorder request parameters in Edge Functions

[skw]

The simplest way to pass a url via request parameter is:

https://aesthetic-malasada-ac66d4.netlify.app/url?https://example.com?a=1&b=2

All is well in Functions:

export default req => new Response(new URL(req.url).search.substring(1))
export const config = { path: "/url" }

https://example.com?a=1&b=2

But in Edge Functions:

https://aesthetic-malasada-ac66d4.netlify.app/url-edge?https://example.com?a=1&b=2

export default req => new Response(new URL(req.url).search.substring(1))
export const config = { path: "/url-edge" }

b=2&https%3A%2F%2Fexample.com%3Fa=1

[hrishikesh]

Mind putting this together in a repo that we can test?

[skw]

I don’t mind. Please do this

[SamO]

Hi @skw Hrishikesh is asking if you can put this information in a repo so we can test this issue.

[skw]

my repo: trsub/netlify-func (github.com)

[hrishikesh]

I don’t see any reordering happening in a minimal reproduction like:

?b=1&a=2

It gets printed in the same order.

However, I wonder what you’re doing exactly.

@skw

?https://example.com?a=1&b=2

This doesn’t seem like a valie query param. It should probably be ?url=https://example.com?a=1&b=2

[skw]

If it is ?url={url} then the url needs to be urlEncoded, which is inconvenient for the user, but if it is ?{url} the user only needs to copy and paste the url.

[skw]

Like this https://corsproxy.io/?https://httpbin.org/get?a=1&b=2

[dig]

Are you building an open proxy @skw?

Recent change: open proxy deprecation

As part of our ethos to build a better web, our team always strive to do what’s best for both our service and others. Therefore, Netlify have recently deprecated the ability to create open proxies. What is an open proxy, I hear you ask, and what’s (possibly) so bad about them? Well, rather than reinvent the wheel, here’s the Wikipedia definition! Who is impacted? Odds are, you are not impacted. We identified only a few dozen customers who had configured an open proxy, whether intentionally or…

Or some other sort of proxying service?

[skw]

Passing the url is just an example. The point is, query string syntax is not well defined, it is not necessarily key-value pairs:

https://en.wikipedia.org/wiki/URL#Syntax

https://en.wikipedia.org/wiki/Query_string#Structure

So it shouldn’t be changed. I should be able to pass 1,2,3 but I can’t pass it in Edge Functions.

Ok in Functions:

https://testfunc.netlify.app/url?1,2,3

1,2,3

But in Edge Functions:

https://testfunc.netlify.app/url-edge?1,2,3

1%2C2%2C3=

[hrishikesh]

Not sure if wikipedia is the best resource, but RFC disagrees: RFC 3986: Uniform Resource Identifier (URI): Generic Syntax (rfc-editor.org)

As long as we’re working good enough according to the general standard, I don’t think this is a valid feature request or a bug report.

@skw

f it is ?url={url} then the url needs to be urlEncoded, which is inconvenient for the user, but if it is ?{url} the user only needs to copy and paste the url.

I disgree with this. Not sure what your use-case is, but I’m not sure why your users need to care about the query params in the URL. In most cases, browsers handle the URL encoding themselves, it’s only the decoding that’s 1 extra step. If you’re building a UI or something, you can anyways show the correct value to the users.

[skw]

What it says is that query is often key-value pairs, but don’t have to be. So other forms should be allowed, right? In this way, the scope of use of Edge Functions can be expanded.

I use this in Functions, but I can’t migrate to Edge Functions due to inconsistency.

[skw]

for example:

• javascript

new URL('https://example.com/?1,2,3').search

• python

urllib.parse.urlsplit('https://example.com/?1,2,3').query

• java

System.out.println(new java.net.URL("https://example.com/?1,2,3").getQuery());

• go

u, _ := url.Parse("https://example.com/?1,2,3")
fmt.Printf(u.RawQuery)

Whoever returns string 1,2,3 does not change it. Because it’s not necessarily key-value pairs.

So Edge Functions changing it confuses me.

[hrishikesh]

@skw

is often key-value pairs

I think it says “often used to carry xyz” in the form of key=value. The often applies to carrying information and not to the form. The form seems to be well-defined.

In any care, what’s the use case here? What makes you go against a standard and try to implement something that can’t be achieved with the standards? The URL needs to be encoded is not a valid argument as that’s how it should be. Websites all over the world use encoded URLs in query params, so I’m interested to know the problem you’re trying to solve here.

[skw]

There are also some that are not standard, like what I mentioned earlier https://corsproxy.io

[skw]

I want to allow users to type the URL themselves, instead of going through the web page, which is sometimes more convenient.

[skw]

It would be nice as long as Edge Functions didn’t parse and stringify query parameters, and instead did nothing like Functions.

I have tried netlify dev without this issue, which only occurs after deployment.

[skw]

@hrishikesh

I think it says “often used to carry xyz” in the form of key=value. The often applies to carrying information and not to the form. The form seems to be well-defined.

Hmmm. I tried various translators, ChatGPT, Google, DeepL, etc., and the results were query components are often used to “carry identifying information in the form of …”. sth is often used to do sth.

[hrishikesh]

I’m sorry to say, but I don’t think I’m still convinced about:

• the use case (as I mentioned, users can still directly type the URL and the browser would handle the URL encoding itself)
• the way you’re handling query params

With that being said, I’m still filing this for the devs to review if this is possible to change. Do note that, this would be low in priority as there are other pressing issues taking preference.

[hrishikesh]

This behaviour has been fixed and standardized across all our services (Functions, Edge Functions, Cloud and CLI). The fix affects the encoding of request.url, when there’s commas in the search query. Consider a request to /products?1,2,3. For reasons, our CDN turns that into /products?1%2C2%2C3. Now with this change rolled out, it’s turned back into /products?1,2,3 in Edge Functions, so devs are seeing the request exactly as it comes in from the browser.

Regarding the original reported issue the devs mentioned the following:

There might be some misunderstanding here on what the expected behaviour is in the example shared:

https://aesthetic-malasada-ac66d4.netlify.app/url?https://example.com?a=1&b=2

You might be expecting url search parameter to get the value https://example.com?a=1&b=2, but that’s not how it works:

new URL("https://aesthetic-malasada-ac66d4.netlify.app/url?https://example.com?a=1&b=2").searchParams
URLSearchParams { 'https://example.com?a' => '1', 'b' => '2' }