502 error when making an api request from the production netlify

gaborhadhazy · July 23, 2023, 2:50pm

Hello guys, I have a production website hosted on netlify. In my app I’m using various types of APIs such as the openAI API and for some reason I got a 502 error when using it. I want to mention that the API worked on localhost and if I’m using the production url which is this: https://student-ai.io/api/quiz-generator as shown on the image.

This is the code:

import { NextApiRequest, NextApiResponse } from "next";
import { Configuration, OpenAIApi } from "openai";

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse
) {
  res.setHeader("Access-Control-Allow-Origin", "*");
  res.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
  res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
  if (req.method !== "POST") {
    res.status(405).json({ error: "Method not allowed" });
    return;
  }

  const { messageContent } = req.body;

  if (!messageContent) {
    res.status(400).json({ error: "Invalid message content" });
    return;
  }

  console.log("messageContent", messageContent);

  try {
    const configuration = new Configuration({
      apiKey: process.env.API_KEY,
    });

    const openaiAPI = new OpenAIApi(configuration);
    const response = await openaiAPI.createChatCompletion(
      {
        model: "gpt-3.5-turbo",
        messages: [
          {
            role: "user",
            content: messageContent,
          },
        ],
      },
      {
        headers: {
          "User-Agent": "Mozilla/5.0 (compatible; MyCoolApp/1.0)",
        },
      }
    );

    console.log("Outside response", response);
    if (response.data.choices && response.data.choices.length > 0) {
      console.log("here");
      console.log("Inside response", response);

      const firstChoice = response.data.choices[0];
      console.log("firstChoice", firstChoice);
      if (firstChoice.message && firstChoice.message.content) {
        const content = firstChoice.message.content;
        res.status(200).json({ content });
      }
    }
  } catch (error) {
    console.error(error);
    res.status(500).json({ error: "An error occurred" });
  }
}

and this is the error that I’m getting:

hrishikesh · July 23, 2023, 5:15pm

As it says, it’s taking longer than 10 seconds. On Pro and above plans, we can raise it to 26 seconds which is the max supported.

gaborhadhazy · July 23, 2023, 5:22pm

But this doesnt solve the problem, and as of now I would not like to pay to make a simple API request on production. Do I have to?

hrishikesh · July 23, 2023, 5:24pm

How does it not? You need more than 10 seconds to do your thing, we’re offering more than 10 seconds for that. Either wrap up within 10 seconds, or switch to Pro to get more than 10 seconds.

Or try this: [Support Guide] Why is my function taking long or timing out?

gaborhadhazy · July 23, 2023, 5:47pm

So If I’d have the pro plan then the whole thing would work, wouldnt it?

hrishikesh · July 23, 2023, 5:48pm

Does your API respond within 10 seconds? If yes, Starter would work. Does it respond within 26 seconds? If yes, any of the other plans would work. If it takes longer than that however, Netlify isn’t the platform for your use-case.

gaborhadhazy · July 23, 2023, 5:53pm

Well I need to have a large respond limit as the requests are relatively slow and they are communicating with openAI API which as you know can take long. I’m beginner in this webhosting stuff, can you recommend me anything that I can potentially go with?

hrishikesh · July 23, 2023, 6:01pm

You can use Netlify Edge Functions: Long-running Edge Functions | Edge Functions on Netlify (edge-functions-examples.netlify.app) where the time limit should not matter as long as you’re simply waiting for the API to send a response.

gaborhadhazy · July 23, 2023, 6:34pm

Does it work like a regular API request but without time limit?

hrishikesh · July 23, 2023, 6:50pm

More or less, yes. You can read more about Edge Functions here: Edge Functions overview | Netlify Docs

You’d basically have to deploy this either as a Next.js middleware or as a standalone Netlify Edge Function. You can’t use this as Next.js API.

gaborhadhazy · July 23, 2023, 8:01pm

I apologise but I don’t fully understand. How would I deploy them and how would this help me?

gaborhadhazy · July 24, 2023, 2:50pm

I have successfuly set up a netlify function, but I’m getting an error when executing the function, firstly it worked locally perfeclty and I was able to deploy the funtion however when calling the function I’ve came accross some errors:

Jul 24, 03:15:34 PM: INIT_START Runtime Version: nodejs:18.v9	Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:7d5f06b69c951da8a48b926ce280a9daf2e8bb1a74fc4a2672580c787d608206Jul 24, 03:15:34 PM: 2023-07-24T14:15:34.501Z	undefined	ERROR	(node:8) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)Jul 24, 03:15:34 PM: 2023-07-24T14:15:34.502Z	undefined	ERROR	Uncaught Exception 	{"errorType":"Runtime.UserCodeSyntaxError","errorMessage":"SyntaxError: Cannot use import statement outside a module","stack":["Runtime.UserCodeSyntaxError: SyntaxError: Cannot use import statement outside a module","    at _loadUserApp (file:///var/runtime/index.mjs:994:17)","    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1029:21)","    at async start (file:///var/runtime/index.mjs:1192:23)","    at async file:///var/runtime/index.mjs:1198:1"]}Jul 24, 03:15:34 PM: 2023-07-24T14:15:34.520Z	undefined	ERROR	Unhandled Promise Rejection 	{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"SyntaxError: Cannot use import statement outside a module","reason":{"errorType":"SyntaxError","errorMessage":"Cannot use import statement outside a module","stack":["/var/task/netlify/edge-functions/index.js:2","    import {createRequire as ___nfyCreateRequire} from \"module\";","    ^^^^^^","","SyntaxError: Cannot use import statement outside a module","    at internalCompileFunction (node:internal/vm:73:18)","    at wrapSafe (node:internal/modules/cjs/loader:1178:20)","    at Module._compile (node:internal/modules/cjs/loader:1220:27)","    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)","    at Module.load (node:internal/modules/cjs/loader:1119:32)","    at Module._load (node:internal/modules/cjs/loader:960:12)","    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)","    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: SyntaxError: Cannot use import statement outside a module","    at process.<anonymous> (file:///var/runtime/index.mjs:1186:17)","    at process.emit (node:events:513:28)","    at emit (node:internal/process/promises:149:20)","    at processPromiseRejections (node:internal/process/promises:283:27)","    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)"]}Jul 24, 03:15:35 PM: 2023-07-24T14:15:35.380Z	undefined	ERROR	(node:8) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)Jul 24, 03:15:35 PM: 2023-07-24T14:15:35.381Z	undefined	ERROR	Uncaught Exception 	{"errorType":"Runtime.UserCodeSyntaxError","errorMessage":"SyntaxError: Cannot use import statement outside a module","stack":["Runtime.UserCodeSyntaxError: SyntaxError: Cannot use import statement outside a module","    at _loadUserApp (file:///var/runtime/index.mjs:994:17)","    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1029:21)","    at async start (file:///var/runtime/index.mjs:1192:23)","    at async file:///var/runtime/index.mjs:1198:1"]}Jul 24, 03:15:35 PM: 2023-07-24T14:15:35.398Z	undefined	ERROR	Unhandled Promise Rejection 	{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"SyntaxError: Cannot use import statement outside a module","reason":{"errorType":"SyntaxError","errorMessage":"Cannot use import statement outside a module","stack":["/var/task/netlify/edge-functions/index.js:2","    import {createRequire as ___nfyCreateRequire} from \"module\";","    ^^^^^^","","SyntaxError: Cannot use import statement outside a module","    at internalCompileFunction (node:internal/vm:73:18)","    at wrapSafe (node:internal/modules/cjs/loader:1178:20)","    at Module._compile (node:internal/modules/cjs/loader:1220:27)","    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)","    at Module.load (node:internal/modules/cjs/loader:1119:32)","    at Module._load (node:internal/modules/cjs/loader:960:12)","    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)","    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: SyntaxError: Cannot use import statement outside a module","    at process.<anonymous> (file:///var/runtime/index.mjs:1186:17)","    at process.emit (node:events:513:28)","    at emit (node:internal/process/promises:149:20)","    at processPromiseRejections (node:internal/process/promises:283:27)","    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)"]}Jul 24, 03:15:35 PM: Unknown application error occurred

my function implementation code is:

// Import Axios at the top of your file
import axios from "axios";

export default async (req, res) => {
  try {
    if (req.method !== "POST") {
      return new Response(JSON.stringify({ error: "Method not allowed" }), {
        status: 405,
        headers: { "Content-Type": "application/json" },
      });
    }

    // Parse the request body as JSON
    const body = await req.json();
    const { messageContent } = body;

    if (!messageContent) {
      return new Response(
        JSON.stringify({ error: "Invalid message content" }),
        {
          status: 400,
          headers: { "Content-Type": "application/json" },
        }
      );
    }

    const apiKey = process.env.OPEN_AI_KEY;
    const model = "gpt-3.5-turbo";

    // Use Axios instead of fetch
    const response = await axios.post(
      "https://api.openai.com/v1/chat/completions",
      {
        model,
        messages: [
          {
            role: "user",
            content: messageContent,
          },
        ],
      },
      {
        headers: {
          Authorization: `Bearer ${apiKey}`,
          "User-Agent": "Mozilla/5.0 (compatible; MyCoolApp/1.0)",
          "Content-Type": "application/json",
        },
      }
    );

    // Access the response data using response.data
    const responseData = response.data;

    if (
      response.status === 200 &&
      responseData.choices &&
      responseData.choices.length > 0
    ) {
      const firstChoice = responseData.choices[0];
      if (firstChoice.message && firstChoice.message.content) {
        const content = firstChoice.message.content;
        return new Response(JSON.stringify({ content }), {
          status: 200,
          headers: { "Content-Type": "application/json" },
        });
      }
    }

    return new Response(
      JSON.stringify({ error: "No response content available" }),
      {
        status: 500,
        headers: { "Content-Type": "application/json" },
      }
    );
  } catch (error) {
    console.error(error);
    return new Response(
      JSON.stringify({ error: `An error occurred, ${error.message}` }),
      {
        status: 500,
        headers: { "Content-Type": "application/json" },
      }
    );
  }
};

export const config = {
  path: "/api/openai-api",
};

These are some additional screenshots:

fool · July 25, 2023, 8:25pm

Seems like “something” is happening but we’re not sure what, with the uncaught exception. Love that you already added the error handling code to show more details, so now, what’s the cause of this?

Cannot use import statement outside a module

Perhaps some googling will turn up the relevant detail? I didn’t find anything definitely correlated when I tried, but articles like this may point you in the right direction: javascript - Node.js: SyntaxError: Cannot use import statement outside a module - Stack Overflow

Another different thing you could try is to activate the Pro trial to see if your initial function was able to run within 26 seconds or not. For Pro trials (free Pro for a month), we can increase your timeout to 26 seconds in case that would be more expedient in your debugging/working around whatever is slow about your API.

(But also consider that our platform is simply not designed for slow API calls - what user wants to wait any number of seconds for a webpage to load?)