Redirect / Proxy set custom user-agent for the request

Support
1

PLEASE help us help you by writing a good post!

  • We need to know your netlify site name. Example: gifted-antelope-58b104.netlify.app
  • DNS issues? Tell us the custom domain, tell us the error message! We can’t help if we don’t know your domain.
  • Build problems? Link or paste the FULL build log & build settings screenshot
  • Did you try Ask Netlify, our generative AI chatbot, before posting? It pulls info from Support Guides and recent solved forums posts.

The better the post - the faster the answer.

Site: middlewood-locks.co.uk

Basic site architechture is the front end is a NUXT build on netlify, and the data is served from a wordpress site hosted on Siteground.

I’ve succesfully set up a redirect acting as a proxy in my netlify.toml file. So when carring out requests on the front end or SSR requests it will query /api/* but retrieve https://middlewood-locks-live.drumbeaters.link/wp-cms/index.php/*

[[redirects]]
  from = "/api/*"
  to = "https://middlewood-locks-live.drumbeaters.link/wp-cms/index.php/:splat"
  status = 200
  headers = {User-Agent = "MiddlewoodLocksFrontEnd"}
  force = true # COMMENT: ensure that we always redirect

I know that the redirect is working from a technical point of view, the only issue is that siteground is blocking the request on the terms that the user-agent of the request is “Go-http-client” and this is linked to spam apps.

I imagine this is for the SSR based requests coming from Netlify, and not the direct frontend requests once loaded.

Is there any way to change this user-agent or solve this issue?

2

It appears that SiteGround is blocking requests from your Nuxt frontend hosted on Netlify due to the default Go-http-client user agent used in server-side rendering (SSR) requests. This is a common issue when the backend perceives such user agents as spammy. To address this, you need to ensure that your SSR requests include a custom User-Agent header.

Here’s how you can resolve this:

  1. Manually Set the User-Agent in Your Nuxt SSR Fetch Requests:

    Netlify’s redirect rules may not always pass custom headers as intended, especially in SSR contexts. Therefore, it’s more reliable to set the User-Agent header directly within your Nuxt application when making SSR fetch requests. You can achieve this by using Nuxt’s $fetch or useFetch utilities and specifying the desired headers.

    Here’s an example of how to set the User-Agent header in an SSR fetch request:

    import { useFetch } from '#app';

export default {
  async asyncData() {
    const { data, error } = await useFetch('https://middlewood-locks-live.drumbeaters.link/wp-cms/index.php/your-endpoint', {
      headers: {
        'User-Agent': 'MiddlewoodLocksFrontEnd'
      }
    });

    if (error.value) {
      console.error('Fetch error:', error.value);
    }

    return { data: data.value };
  }
};

By specifying the User-Agent header in your fetch requests, you ensure that SiteGround recognizes and allows the requests from your Nuxt application.

  1. Verify the Custom Header:

    After implementing the custom header, it’s essential to verify that your requests are being sent with the correct User-Agent. You can use tools like cURL to test the requests:

    curl -A "MiddlewoodLocksFrontEnd" https://middlewood-locks.co.uk/api/some-endpoint

If the request is successful and not blocked by SiteGround, it indicates that the custom User-Agent header is being applied correctly.

  1. Consult SiteGround Support if Issues Persist:

    If, after setting the custom User-Agent header, you continue to experience blocking issues, consider reaching out to SiteGround’s support team. Provide them with details about your application and the steps you’ve taken to set the User-Agent header. They may offer insights or solutions tailored to their server configurations.

Additional Considerations:

  • Netlify Redirects and Headers: While Netlify allows setting custom headers in redirect rules, these headers are typically applied to responses sent to the client, not to proxied requests. Therefore, relying solely on Netlify’s redirect rules to set the User-Agent header for SSR requests may not be effective. Directly setting the header within your Nuxt application ensures that it is included in the outgoing requests to your backend.

  • Security Implications: When modifying headers, ensure that the changes do not inadvertently expose sensitive information or introduce security vulnerabilities. Always validate and sanitize inputs and headers as needed.

By implementing the custom User-Agent header directly within your Nuxt application’s SSR fetch requests, you should be able to resolve the blocking issue with SiteGround and ensure seamless communication between your frontend and backend services.