Netlify function debug help

Support
,
#1

I forked netlify functions repo and added my own code (this is my additional code). I also deployed the code onto Netlify. The test is located at the bottom of the Netlify page. In short, I added 2 Netlify Functions

  1. Original: Uses ytdl-core and collect youtube video info, based on the VideoID obtained in the query parameter (e.g. .netlify/functions/youtube?videoID=UVV80kTudSM)

  2. Modded: Exactly the same API as the one above, with addition JSON post-processing

Test Case 1:

  • Call modded API, with the parameter UVV80kTudSM, and run it multiple times (browser refresh). The API fails half of the time.

Here is the attach the Netlify Function log, after calling the API multiple times:

8:30:33 AM: Duration: 7.92 ms Memory Usage: 96 MB
8:30:36 AM: Duration: 721.67 ms Memory Usage: 96 MB
8:30:39 AM: Duration: 10.96 ms Memory Usage: 96 MB
8:31:33 AM: Duration: 641.21 ms Memory Usage: 96 MB
8:31:36 AM: Duration: 4.93 ms Memory Usage: 96 MB
8:31:46 AM: Duration: 697.44 ms Memory Usage: 96 MB

The long-short-long-short API duration actually synced with the pass/fail situation. It seems to me that the reason of the short duration is because the Netlify function is not being executed fully.

Test Case 2:

  1. Run the Modded API, with parameter UVV80kTudSM
  2. Run the Modded API, with parameter DylgPyQxDiI
  3. Loop through Step 1 and Step 2

When I run the above sequence, the Youtube (Full) API works fine. The API doesn’t work fine only when the same API (with the same parameter) is being called again. Therefore I am not sure if the problem comes from my code or the behavior (caching?) of the Netlify function.

Thanks again for your time.

#2

Our system will cache responses based on the Cache-Control header. You could try adding the following in your callback function’s header object:

cache-control: no-cache no-store max-age=0 must-revalidate

Let me know if that helps.

#3

Hi @Dennis,

I added the cache-control and here is my commit and here is the API code. It is re-deployed on Netlify.

But when I rerun Test case 1 above, the duration behavior didn’t change - it is still one long and one short, I would assume that the short duration is caused by Netlify Functions replying using cache.

The problem here is the caching behavior breaks line 12 of my code. In contrast, the original API (code, with the same parameter, would not fail on re-run.

#6

I can confirm that we are not returning a cached response for your function. What you are seeing is a caching issue but rather a cold-start one. The longer requests are likely when a function needs to cold-start and the short requests one to an already spun-up function.

I suspect that the issue is in how you are calling ‘chooseFormats’ (https://github.com/fent/node-ytdl-core#ytdlchooseformatformats-options). But you’ll likely need to figure that out yourself.

#8

Thanks

Hmm I tried the API code locally and I don’t see any problem, that is strange.

Still, the API calling duration makes me feel extremely curious. Also if you run the Test Case #2 (if caching is not being observed, then the long short duration behavior should not exist when the API parameter is changed) above, actually there is no issue, also according to the log the duration value keeps flat.

#9

As I mentioned, I’ve reviewed our logs with a colleague and can definitely confirm that we are not responding with a cached response. I believe the issue is directly related to the difference in your API calls. I’m not familiar with that package you are using so I won’t be much help with that.

#10

@Dennis. Thanks for the reply.

I modified hello_fetch function (test) and didn’t get any fetch error problem. I guess the problem may not come from Netlify Function.

But honestly I still couldn’t explain why there is no problem when my code is executed locally but problem exist when it is deployed. Very confusing.

#11

Testing locally won’t always replicate how things work live, since there are a lot more factors when the function is deployed. Even the Node version could be different. You could try setting the node runtime version of your deployed Netlify function to match what you use locally by following: https://docs.netlify.com/functions/build-with-javascript/#runtime-settings.

#12

@Dennis

Thanks for your suggestion. Will have a try.