Problem with CAIXA route

Support
,
1

Hi!

I am having a problem with my getLotofacilContests schedule function.

For some time, it worked perfectly, but nowadays, it starts to return errors.

The error is that I am receiving a 401 or 403 from the CAIXA website link.

But if I run the code on my local machine, it works perfectly!

Can you help me make the function work properly again? It is hard to make it run every day on my local machine.

My website:
https://lotoproweb.com.br/

Page for which I took the URL:
https://loterias.caixa.gov.br/Paginas/Lotofacil.aspx

The request that I need to access every day is:

curl 'https://servicebus2.caixa.gov.br/portaldeloterias/api/lotofacil/3131' \
  -H 'accept: application/json, text/plain, */*' \
  -H 'accept-language: pt-BR,pt;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6' \
  -H 'origin: https://loterias.caixa.gov.br' \
  -H 'priority: u=1, i' \
  -H 'referer: https://loterias.caixa.gov.br/' \
  -H 'sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Microsoft Edge";v="126"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'sec-fetch-dest: empty' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-site: same-site' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0'

Error logs:

Jun 18, 10:00:17 PM: 811dea1f  8.42079 0.5 8.27027 0.5C8.11974 0.5 7.97538 0.559797 7.86894 0.666237C7.7625 0.772676 7.7027 0.917039 7.7027 1.06757V2.01351H4.29729V1.06757C4.29729 0.917039 4.2375 0.772676 4.13106 0.666237C4.02462 0.559797 3.88025 0.5 3.72973 0.5C3.5792 0.5 3.43484 0.559797 3.3284 0.666237C3.22196 0.772676 3.16216 0.917039 3.16216 1.06757V2.01351H2.21621C1.66428 2.01351 1.13494 2.23277 0.744666 2.62305C0.354388 3.01333 0.135132 3.54266 0.135132 4.09459V12.4189C0.135132 12.9709 0.354388 13.5002 0.744666 13.8905C1.13494 14.2807 1.66428 14.5 2.21621 14.5H9.78378C10.3357 14.5 10.865 14.2807 11.2553 13.8905C11.6456 13.5002 11.8649 12.9709 11.8649 12.4189V4.09459C11.8649 3.54266 11.6456 3.01333 11.2553 2.62305C10.865 2.23277 10.3357 2.01351 9.78378 2.01351ZM2.21621 3.14865H3.16216V4.09459C3.16216 4.24512 3.22196 4.38949 3.3284 4.49593C3.43484 4.60237 3.5792 4.66216 3.72973 4.66216C3.88025 4.66216 4.02462 4.60237 4.13106 4.49593C4.2375 4.38949 4.29729 4.24512 4.29729 4.09459V3.14865H7.7027V4.09459C7.7027 4.24512 7.7625 4.38949 7.86894 4.49593C7.97538 4.60237 8.11974 4.66216 8.27027 4.66216C8.42079 4.66216 8.56516 4.60237 8.6716 4.49593C8.77804 4.38949 8.83783 4.24512 8.83783 4.09459V3.14865H9.78378C10.0347 3.14865 10.2753 3.24831 10.4527 3.42571C10.6301 3.60311 10.7297 3.84371 10.7297 4.09459V6.17568H1.27027V4.09459C1.27027 3.84371 1.36993 3.60311 1.54733 3.42571C1.72473 3.24831 1.96533 3.14865 2.21621 3.14865ZM9.78378 13.3649H2.21621C1.96533 13.3649 1.72473 13.2652 1.54733 13.0878C1.36993 12.9104 1.27027 12.6698 1.27027 12.4189V7.31081H10.7297V12.4189C10.7297 12.6698 10.6301 12.9104 10.4527 13.0878C10.2753 13.2652 10.0347 13.3649 9.78378 13.3649Z" fill="#F4F4F4"/> </svg> <span class="card-details__value"> 2024-06-19 01:00:18 </span> </p><hr/> <p class="card-details__row"> <span class="card-details__label">Request ID</span> <span class="card-details__value">1d75debb0dd145d96d4bdfae808ad79f-1f117607</span> </p><p class="card-details__row"> <span class="card-details__label">Status Code</span> <span class="card-details__value">403</span> </p><p class="card-details__row"> <span class="card-details__label">POP</span> <span class="card-details__value">EQN</span> </p></div></main> <footer> <svg width="80px" height="20px" viewBox="0 0 120 24" fill="none" xmlns="http://www.w3.org/2000/svg" class="h-[26px]" > <path fill-rule="evenodd" clip-rule="evenodd" d="M68.8751 0L64.3906 23.4146H87.711L92.1955 0H68.8751ZM72.5179 4.42392H86.8522L84.0623 18.9907H69.728L72.5179 4.42392Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M60.0001 0L55.6094 23.4146H59.9992L64.3899 0H60.0001Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M24.3824 0L0.654522 19.9761L0 23.4146H3.41853L21.7987 7.94855L18.8361 23.4146H23.3197L27.8049 0H24.3824Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M115.516 0L113.526 10.3871L117.148 14.8895L120 0H115.516ZM96.6798 0L92.1953 23.4146H96.6815L99.755 7.36298L112.711 23.4146H115.516L116.069 20.5482L99.4841 0H96.6798Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M32.2893 0L31.4419 4.42401H46.9283L28.4682 19.9674L27.8047 23.4146H51.125L51.9724 18.9906H36.4861L54.9502 3.44385L55.6096 0H32.2893Z" fill="#F3652B" ></path> </svg> <div class="footer__text"> © Azion Technologies, Inc or its affiliates. All rights reserved. </div></footer> </div></body></html>
Jun 18, 10:00:17 PM: 811dea1f ERROR  Invoke Error 	{"errorType":"SyntaxError","errorMessage":"Unexpected token '<', \"<!DOCTYPE \"... is not valid JSON","stack":["SyntaxError: Unexpected token '<', \"<!DOCTYPE \"... is not valid JSON","    at JSON.parse (<anonymous>)","    at Runtime.handler (/var/task/netlify/functions/getLotofacilContests.js:66:26)","    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"]}
Jun 18, 10:00:18 PM: 811dea1f Duration: 798.74 ms	Memory Usage: 118 MB	Init Duration: 519.91 ms
Jun 18, 10:00:19 PM: 9ffae77f 0.666237C8.56516 0.559797 8.42079 0.5 8.27027 0.5C8.11974 0.5 7.97538 0.559797 7.86894 0.666237C7.7625 0.772676 7.7027 0.917039 7.7027 1.06757V2.01351H4.29729V1.06757C4.29729 0.917039 4.2375 0.772676 4.13106 0.666237C4.02462 0.559797 3.88025 0.5 3.72973 0.5C3.5792 0.5 3.43484 0.559797 3.3284 0.666237C3.22196 0.772676 3.16216 0.917039 3.16216 1.06757V2.01351H2.21621C1.66428 2.01351 1.13494 2.23277 0.744666 2.62305C0.354388 3.01333 0.135132 3.54266 0.135132 4.09459V12.4189C0.135132 12.9709 0.354388 13.5002 0.744666 13.8905C1.13494 14.2807 1.66428 14.5 2.21621 14.5H9.78378C10.3357 14.5 10.865 14.2807 11.2553 13.8905C11.6456 13.5002 11.8649 12.9709 11.8649 12.4189V4.09459C11.8649 3.54266 11.6456 3.01333 11.2553 2.62305C10.865 2.23277 10.3357 2.01351 9.78378 2.01351ZM2.21621 3.14865H3.16216V4.09459C3.16216 4.24512 3.22196 4.38949 3.3284 4.49593C3.43484 4.60237 3.5792 4.66216 3.72973 4.66216C3.88025 4.66216 4.02462 4.60237 4.13106 4.49593C4.2375 4.38949 4.29729 4.24512 4.29729 4.09459V3.14865H7.7027V4.09459C7.7027 4.24512 7.7625 4.38949 7.86894 4.49593C7.97538 4.60237 8.11974 4.66216 8.27027 4.66216C8.42079 4.66216 8.56516 4.60237 8.6716 4.49593C8.77804 4.38949 8.83783 4.24512 8.83783 4.09459V3.14865H9.78378C10.0347 3.14865 10.2753 3.24831 10.4527 3.42571C10.6301 3.60311 10.7297 3.84371 10.7297 4.09459V6.17568H1.27027V4.09459C1.27027 3.84371 1.36993 3.60311 1.54733 3.42571C1.72473 3.24831 1.96533 3.14865 2.21621 3.14865ZM9.78378 13.3649H2.21621C1.96533 13.3649 1.72473 13.2652 1.54733 13.0878C1.36993 12.9104 1.27027 12.6698 1.27027 12.4189V7.31081H10.7297V12.4189C10.7297 12.6698 10.6301 12.9104 10.4527 13.0878C10.2753 13.2652 10.0347 13.3649 9.78378 13.3649Z" fill="#F4F4F4"/> </svg> <span class="card-details__value"> 2024-06-19 01:00:19 </span> </p><hr/> <p class="card-details__row"> <span class="card-details__label">Request ID</span> <span class="card-details__value">d9c425602b081c7a711e15980e338c83-1f117607</span> </p><p class="card-details__row"> <span class="card-details__label">Status Code</span> <span class="card-details__value">403</span> </p><p class="card-details__row"> <span class="card-details__label">POP</span> <span class="card-details__value">EQN</span> </p></div></main> <footer> <svg width="80px" height="20px" viewBox="0 0 120 24" fill="none" xmlns="http://www.w3.org/2000/svg" class="h-[26px]" > <path fill-rule="evenodd" clip-rule="evenodd" d="M68.8751 0L64.3906 23.4146H87.711L92.1955 0H68.8751ZM72.5179 4.42392H86.8522L84.0623 18.9907H69.728L72.5179 4.42392Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M60.0001 0L55.6094 23.4146H59.9992L64.3899 0H60.0001Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M24.3824 0L0.654522 19.9761L0 23.4146H3.41853L21.7987 7.94855L18.8361 23.4146H23.3197L27.8049 0H24.3824Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M115.516 0L113.526 10.3871L117.148 14.8895L120 0H115.516ZM96.6798 0L92.1953 23.4146H96.6815L99.755 7.36298L112.711 23.4146H115.516L116.069 20.5482L99.4841 0H96.6798Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M32.2893 0L31.4419 4.42401H46.9283L28.4682 19.9674L27.8047 23.4146H51.125L51.9724 18.9906H36.4861L54.9502 3.44385L55.6096 0H32.2893Z" fill="#F3652B" ></path> </svg> <div class="footer__text"> © Azion Technologies, Inc or its affiliates. All rights reserved. </div></footer> </div></body></html>
Jun 18, 10:00:19 PM: 9ffae77f ERROR  Invoke Error 	{"errorType":"SyntaxError","errorMessage":"Unexpected token '<', \"<!DOCTYPE \"... is not valid JSON","stack":["SyntaxError: Unexpected token '<', \"<!DOCTYPE \"... is not valid JSON","    at JSON.parse (<anonymous>)","    at Runtime.handler (/var/task/netlify/functions/getLotofacilContests.js:66:26)","    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"]}
Jun 18, 10:00:20 PM: 9ffae77f Duration: 352.09 ms	Memory Usage: 133 MB
Jun 18, 10:00:21 PM: e730239d 0.666237C8.56516 0.559797 8.42079 0.5 8.27027 0.5C8.11974 0.5 7.97538 0.559797 7.86894 0.666237C7.7625 0.772676 7.7027 0.917039 7.7027 1.06757V2.01351H4.29729V1.06757C4.29729 0.917039 4.2375 0.772676 4.13106 0.666237C4.02462 0.559797 3.88025 0.5 3.72973 0.5C3.5792 0.5 3.43484 0.559797 3.3284 0.666237C3.22196 0.772676 3.16216 0.917039 3.16216 1.06757V2.01351H2.21621C1.66428 2.01351 1.13494 2.23277 0.744666 2.62305C0.354388 3.01333 0.135132 3.54266 0.135132 4.09459V12.4189C0.135132 12.9709 0.354388 13.5002 0.744666 13.8905C1.13494 14.2807 1.66428 14.5 2.21621 14.5H9.78378C10.3357 14.5 10.865 14.2807 11.2553 13.8905C11.6456 13.5002 11.8649 12.9709 11.8649 12.4189V4.09459C11.8649 3.54266 11.6456 3.01333 11.2553 2.62305C10.865 2.23277 10.3357 2.01351 9.78378 2.01351ZM2.21621 3.14865H3.16216V4.09459C3.16216 4.24512 3.22196 4.38949 3.3284 4.49593C3.43484 4.60237 3.5792 4.66216 3.72973 4.66216C3.88025 4.66216 4.02462 4.60237 4.13106 4.49593C4.2375 4.38949 4.29729 4.24512 4.29729 4.09459V3.14865H7.7027V4.09459C7.7027 4.24512 7.7625 4.38949 7.86894 4.49593C7.97538 4.60237 8.11974 4.66216 8.27027 4.66216C8.42079 4.66216 8.56516 4.60237 8.6716 4.49593C8.77804 4.38949 8.83783 4.24512 8.83783 4.09459V3.14865H9.78378C10.0347 3.14865 10.2753 3.24831 10.4527 3.42571C10.6301 3.60311 10.7297 3.84371 10.7297 4.09459V6.17568H1.27027V4.09459C1.27027 3.84371 1.36993 3.60311 1.54733 3.42571C1.72473 3.24831 1.96533 3.14865 2.21621 3.14865ZM9.78378 13.3649H2.21621C1.96533 13.3649 1.72473 13.2652 1.54733 13.0878C1.36993 12.9104 1.27027 12.6698 1.27027 12.4189V7.31081H10.7297V12.4189C10.7297 12.6698 10.6301 12.9104 10.4527 13.0878C10.2753 13.2652 10.0347 13.3649 9.78378 13.3649Z" fill="#F4F4F4"/> </svg> <span class="card-details__value"> 2024-06-19 01:00:21 </span> </p><hr/> <p class="card-details__row"> <span class="card-details__label">Request ID</span> <span class="card-details__value">159ecd1a272b524cfbf2a9bb03b47b74-1f117607</span> </p><p class="card-details__row"> <span class="card-details__label">Status Code</span> <span class="card-details__value">403</span> </p><p class="card-details__row"> <span class="card-details__label">POP</span> <span class="card-details__value">EQN</span> </p></div></main> <footer> <svg width="80px" height="20px" viewBox="0 0 120 24" fill="none" xmlns="http://www.w3.org/2000/svg" class="h-[26px]" > <path fill-rule="evenodd" clip-rule="evenodd" d="M68.8751 0L64.3906 23.4146H87.711L92.1955 0H68.8751ZM72.5179 4.42392H86.8522L84.0623 18.9907H69.728L72.5179 4.42392Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M60.0001 0L55.6094 23.4146H59.9992L64.3899 0H60.0001Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M24.3824 0L0.654522 19.9761L0 23.4146H3.41853L21.7987 7.94855L18.8361 23.4146H23.3197L27.8049 0H24.3824Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M115.516 0L113.526 10.3871L117.148 14.8895L120 0H115.516ZM96.6798 0L92.1953 23.4146H96.6815L99.755 7.36298L112.711 23.4146H115.516L116.069 20.5482L99.4841 0H96.6798Z" fill="#F3652B" ></path> <path fill-rule="evenodd" clip-rule="evenodd" d="M32.2893 0L31.4419 4.42401H46.9283L28.4682 19.9674L27.8047 23.4146H51.125L51.9724 18.9906H36.4861L54.9502 3.44385L55.6096 0H32.2893Z" fill="#F3652B" ></path> </svg> <div class="footer__text"> © Azion Technologies, Inc or its affiliates. All rights reserved. </div></footer> </div></body></html>
Jun 18, 10:00:21 PM: e730239d ERROR  Invoke Error 	{"errorType":"SyntaxError","errorMessage":"Unexpected token '<', \"<!DOCTYPE \"... is not valid JSON","stack":["SyntaxError: Unexpected token '<', \"<!DOCTYPE \"... is not valid JSON","    at JSON.parse (<anonymous>)","    at Runtime.handler (/var/task/netlify/functions/getLotofacilContests.js:66:26)","    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"]}
Jun 18, 10:00:21 PM: e730239d Duration: 238.97 ms	Memory Usage: 138 MB
2

Netlify has no direct control over what response you receive from the other server.

As you’re probably aware:
401 is Unauthorized
403 is Forbidden

How are you making the request within the function?
Is it identical to the curl?

3

This is how I am making the request:

const fetch = (...args) => import('node-fetch').then(({ default: fetch }) => fetch(...args));

// 3000 is just a contest number example
const responseContent = await (await fetch("https://servicebus2.caixa.gov.br/portaldeloterias/api/lotofacil/" + (3000), {
      "headers": {
        "origin": "https://loterias.caixa.gov.br",
        "accept": "application/json, text/plain, */*",
        "accept-language": "pt-BR,pt;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6",
        "sec-ch-ua": "\"Microsoft Edge\";v=\"123\", \"Not:A-Brand\";v=\"8\", \"Chromium\";v=\"123\"",
        "sec-ch-ua-mobile": "?0",
        "sec-ch-ua-platform": "\"Windows\"",
        "sec-fetch-dest": "empty",
        "sec-fetch-mode": "cors",
        "sec-fetch-site": "same-site"
      },
      "referrer": "https://loterias.caixa.gov.br/",
      "referrerPolicy": "strict-origin-when-cross-origin",
      "body": null,
      "method": "GET",
      "mode": "cors",
      "credentials": "omit"
    })).text();

I think the problem is not with the code.

I think it is something about the machine that is sending the request, because it is a public route, and no auth is required.

Is there a way that Netlify can help me in this case? Maybe changing the environment in which the function is running? A Brazilian one, maybe?

4

I doubt there is anything wrong with the machine making the request.
Auth being required also has no real bearing on it.
The destination server can respond to any request with any response/status that it likes.
Only that server would know precisely why it’s rejecting your requests.

Perhaps, I can’t advise on that though, as I don’t work for them.
You’ll need to wait for one of their staff to chime in.

The origin of the request certainly may be what is causing it to get rejected, it’d be worth trying.

Provided that scheduled functions run in the same regions as regular functions, then SĂŁo Paulo is apparently a region that you can change to via self-service.

You can see the information and regions here:
https://docs.netlify.com/functions/optional-configuration/#region

sa-east-1 - South America (SĂŁo Paulo)

You need to be on at least a Pro Plan to take advantage of it.

5

I have tried this, but it doesn’t work :cry:

6

In what way?

  • Do you have a Pro Plan?
  • If yes, did the function region not change?
  • If the region did change, did the caixa website still return 401/403 to your function?
7

Yes, same result…

I don’t know what else can I do to fix it…

And sorry for the delay.

8

@SamO @hillary @hrishikesh or someone else from Netlify employees team.

Can you help me with this?

9

I asked several very specific questions, but you’ve answered with effectively a single “Yes”.

Can you answer the questions I asked?

Doing so will help everyone understand exactly what the result has been.

1 Like
10

I can only agree with @nathanmartin. Did you make an attempt to contact the server returning the 401/403? Not sure how we can help here. We cannot control the response sent by the other server.

1 Like
11

Sorry for the long delay.

The problem was with the IP of my code runner; the CAIXA API was blocking it for some reason.

The solution was to hire a Brazilian server to run the function.

Thanks for your attention and support.

1 Like
12

Hi everyone,

I believe I’ve found the solution for my case, but I think I’ve also identified the root cause of the problem.

To access Brazilian government websites, the IP address needs to fall within a specific range managed by LACNIC and NICBR.

The issue was that the IP of the Netlify server was not within the allowed range, which is why I was facing this problem.

I’m not sure if Netlify can provide a solution for this, but I hope this information helps the Netlify team in any way.