Netlify dev request protocol is incorrectly set to https

[rachitpant]

There is a nice article on how to use express with netlify.

Express on Netlify

However if you try to run the example shared above, and inspect req.protocol , for some reason req.protocol is https . This caues issues with “express” based packages that depend on req.protocol. One workaround for this is to set request header on redirect.

[[redirects]]
  from = "/somepath/*"
  to = "/.netlify/functions/first_function/:splat"
  status = 200
  force = true
  [redirects.headers]
    X-Forwarded-Proto = "http"

[[redirects]]
  from = "/another-path/*"
  to = "/.netlify/functions/another-function/:splat"
  status = 200
  force = true
  [redirects.headers]
    X-Forwarded-Proto = "http"

And then within the express app we can do app.set('trust proxy'). As explained in the linked express js documentation , this will set the req. protocol based on the value of X-Forwarded-Proto.

However there are multiple issues with this approach. The user will need to most likely set this for all functions and will have to repeat the config to set redirect header for each function.

  [redirects.headers]
    X-Forwarded-Proto = "http"

Second issue. User most likely only needs to manually set this for dev environment. ( netlify dev ). Now there can be workarounds for this too like creating a seperate toml file dev environment and then using it

netlify dev -c "netlify.dev.toml"

Another workaround could be to check the value of process.env.NETLIFY_LOCAL and conditionally set app.set('trust proxy') only if we are working in local mode. ( Assuming user has gone through the hassle of repeatedly setting X-Forwarded-Proto = "http" for each function redirect)

My suggestion : netlify proxy should check the local url , or whether we are setting tls certicate to determine if the local requests are being served on a “http” or “https” url , and accordingly set the value of header X-Forwarded-Proto . I am guessing 95% of the time , when a developer is running netlify dev locally they are not going to “mimic” prod environment and set the tls certificate. Setting X-Forwarded-Proto to “http” by default in dev mode when the requests are served on http , would save a lot of developer headaches as explained above.-- And also its incorrect to have req.protocol in express be “https” when the request is served on http.

[rachitpant]

I noticed another user reported facing a similar issue with netlify edge functions and has the same workaround of setting the header x-forwarded-proto.
Still believe that these issues with both regular serverless function and edge function will be resolved if netlify dev proxy intelligently ( and correctly ) sets the value of x-forwarded-proto. I am referring to the below post.

Using Ngrok with Netlify Dev & Edge Functions

I am trying to debug a function that gets executed with a webhook from a third party app. I am using ngrok as a proxy to relay the webhook to localhost. If I disable all edge functions, all works fine. But once Deno is in the mix, any request coming from ngrok crashes, but making the same requests locally on localhost runs fine. I was able to identify the specific line of code that’s causing the issue. The origin request sent by Ngrok contains both x-forwarded-proto and x-forwarded-for headers…

[hrishikesh]

Sounds like something specific to Express, as it doesn’t seem to be happening at least for standard Functions. I have a fastify app setup: hrishikesh-k/netlify-support (github.com) and logged req.protocol which gave me http.

If this was a Nelify CLI issue, it would have happened with other libraries as well.

Or maybe I’m missing some specific configuration that would help me reproduce this?

[rachitpant]

It might be specific to express. Express correctly interprets the request protocol if the request directly goes it. However when a request is forwarded to it through netlify dev, the value of req.protocol becomes https.

I can confirm this because when i ran this exact same code without netlify , the req. protocol is http. The problem happens as soon as i wrap the api in serverless-http and use netlify dev to serve it. Then req.protocol becomes https. Might be easier to try this on the smaller “express on netlify” example provided on netlify docs.

I think this is what happens. Netlify Dev proxies requests to your local Express server through a secure HTTPS connection, even if the original request from the client was over HTTP. ( probably to mimic production environment ). If i inspect the headers, i notice netlify sets the x-forwarded-for header , but does not set the x-forwarded-proto header.

( The x-forwarded-proto you see above is because i set it in the redirects.header in the netlify.toml file. ). If i do not do this and inspect the headers , there is no x-forwarded-proto set.

I cannot comment on why fastify interprets this differently. However express is very popular and you can inspect the req.protocol in the code base in the netlify express documentation that i linked above. Since netlify is setting x-forwarded-for , why not go ahead and also populate x-forwarded-proto. I think netlify prides itself on developer advocacy and making developer lives easy. This is not an end-of-world , and we can set this header ourselves , but still believe the netlify dev server should correctly set x-forwarded-proto before fowarding the request to the function handler.

[rachitpant]

@hrishikesh Any comments on this. Should i open an issue on github.

[hrishikesh]

This is where Express determines the protocol: express/lib/request.js at d97d79ed9a25099ec4f0537ad8bf2a9378350a6b · expressjs/express · GitHub and this is where Fastify does it: fastify/lib/request.js at f9f0c9f17cf328be4205ecb7b21fc0f5e404b6c6 · fastify/fastify · GitHub

Fastify seems to be checking for encrypted socket in the absence of x-forwarded-proto header which results in the correct value. Express has some config with trusting proxies: Express behind proxies (expressjs.com) (as you have mentioned).

As for the CLI, we’re setting the protocol here: cli/src/lib/functions/server.ts at main · netlify/cli (github.com) and this is where we set the x-forwarded-for header: cli/src/utils/proxy.ts at main · netlify/cli (github.com)

If you try to log the protocol here: cli/src/lib/functions/server.ts at main · netlify/cli (github.com), it correctly gets logged as http and no where in the CLI codebase I could see an evidence of:

@rachitpant

Netlify Dev proxies requests to your local Express server through a secure HTTPS connection, even if the original request from the client was over HTTP. ( probably to mimic production environment ).

The CLI seems to be forwarding whatever headers it’s getting for the most part so it’s likely that it doesn’t get the x-forwarded-proto header from anywhere (unless you set it in the netlify.toml) and we’re not explicity (un)setting it anywhere.

The weird part is, if I add the x-forwarded-proto header here: cli/src/utils/proxy.ts at main · netlify/cli (github.com), Express still logs it incorrectly as displayed here:

The code as you can imagine is this:

import express, {Router} from 'express'
import serverless from 'serverless-http'
const api = express()
const router = Router()
router.get('/', (req, res) => {
  res.json({
    headers: req.headers,
    protocol: req.protocol
  })
})
api.use('/.netlify/functions/api', router)
export const handler = serverless(api)

I’m modifying the CLI code directly in the node_modules folder:

but even after forcing the header to http, it’s reading it as https.

I’d file an issue to set it, but not sure if that could cause some other unexpected issues. Also, it’ll be low priority so do not expect anything to change soon.