Netlify Blobs: accessing from function (inside SvelteKit app) requiring explicit token, siteId, deployId

I’m exploring using Netlify Blobs (which I’m aware is in Beta) within my site.

In a local build plugin, I generate a pile of JSON, and set it to a deploy-specific store (using getDeployStore. In this plugin, I have to pass in these credentials to make it work:

  const store = getDeployStore({
    siteID: constants.SITE_ID,
    token: constants.NETLIFY_API_TOKEN,
    deployID: process.env.DEPLOY_ID,
  });
  await store.setJSON('posts', postData);

This works, and populates the store at build time!

I then attempt to access it like so:

    const store = getDeployStore();
    const unsorted:Post[] = await store.get('posts', {type: 'json'});

I’m building a site in SvelteKit; this code gets translated to a Netlify Function, where it runs. If I run the above function, I get the following error:

Runtime.UnhandledPromiseRejection - MissingBlobsEnvironmentError: The environment has not been configured to use Netlify Blobs. To use it manually, supply the following properties when creating a store: deployID

The documentation suggests getDeployStore should Just Work™, but it appears to rely on process.env being available to the function.

Now: my gut says that because I’m running Sveltekit with adapter-netlify, and the build process is managed by Vite, process.env is not available; instead, I should import environment variables from SvelteKit’s own $env/dynamic/private or similar.

If I import DEPLOY_ID from $env/dynamic/private, and pass it by hand into getDeployStore, I now get a new error:

Runtime.UnhandledPromiseRejection - MissingBlobsEnvironmentError: The environment has not been configured to use Netlify Blobs. To use it manually, supply the following properties when creating a store: siteID, token

siteID is easily found in the environment variables, but it appears the code would also like me to supply a full-access Netlify Token. If I generate a full-access PAT, store it as a dedicated environment variable, and then use it in the function to configure my store… everything works. But this feels like a sledgehammer to crack a nut.

How can I specify the token manually inside my function - it feel slike it should “magically” be available inside the function context, and yet it’s unclear how to retrieve it.

Is this happening in Netlify CLI or on the hosted app? There’s already an internal issue to remove this error from the CLI.

This is an error logged to logs/browser when the function is called. It doesn’t have access to the API key, I believe because process.env has been stripped by Vite.

I can make this work by including a PAT manually, but that feels like the incorrect approach based on all documentation.

Thank you for sharing, I’ve passed this on to the devs.

1 Like

The only clue I have - which may be useful to the devs, @hrishikesh - is that the environment variable NETLIFY_BLOBS_CONTEXT appears to be unavailable at both build and runtime for these functions; this line in the blobs library suggests it (or the variable found in globalThis is required), and whenever I build or run the site, this always resolves to undefined.

OK, I have a bit more info that might help.

  • my site is built using SvelteKit’s adapter-netlify. This builds a bunch of pages as static, and turns the dynamic pages into Netlify Functions. (The details of this, which turn out to be important, are something I’m vague on right now).
  • As a test, I made a SvelteKit endpoint that would log environment variables - this endpoint would get turned into a Netlify Function at build time.
  • I also made a manual function in netlify/functions to do the same.

There are two environment variables the function in netlify/functions has available to it that the SvelteKit endpoint does not: NETLIFY_PURGE_API_TOKEN and NETLIFY_BLOBS_CONTEXT.

So that would explain why Blobs aren’t working inside the SvelteKit-generated function; the context is entirely missing. What I don’t currently understand is why those environment variables are missing: I assumed both functions would be running inside the same environment.

I’ll see if I can reach out to SK maintainers to work out what’s going on.

Thanks for the deep-dive. I have passed it on to the devs.

I have the same issue with Next.js.
Any updates?

According to blobs/src/environment.ts at main · netlify/blobs · GitHub there should be NETLIFY_BLOBS_CONTEXT variable.

  1. I have Next.js website.
  2. I have API page route
export default function handler(req, res) {

  res
    .status(200)
    .json({
      test: 'test',
      environment: process.env.NODE_ENV,
      env: JSON.stringify(process.env),
      netlify_blobs: process.env.NETLIFY_BLOBS,
      test2: 'test2',
    });
}

And there is no NETLIFY_BLOBS variable in the output…

{
  "AWS_LAMBDA_FUNCTION_VERSION": "$LATEST",
  "AWS_SESSION_TOKEN": "....",
  "LD_LIBRARY_PATH": "/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib",
  "LAMBDA_TASK_ROOT": "/var/task",
  "AWS_LAMBDA_LOG_GROUP_NAME": "/aws/lambda/d8b33cea294a7d99410f6782b13a8148dcc31166c053ccc8d289564ad8e9bc2b",
  "AWS_LAMBDA_LOG_STREAM_NAME": "2024/02/22/[$LATEST]d1a7fd63a236444c83d8cc9ed106062e",
  "AWS_LAMBDA_RUNTIME_API": "127.0.0.1:9001",
  "AWS_EXECUTION_ENV": "AWS_Lambda_nodejs18.x",
  "SITE_ID": "19c948ef-1830-429c-95af-0fde50f897dc",
  "AWS_LAMBDA_FUNCTION_NAME": "d8b33cea294a7d99410f6782b13a8148dcc31166c053ccc8d289564ad8e9bc2b",
  "AWS_XRAY_DAEMON_ADDRESS": "169.254.79.129:2000",
  "PATH": "/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin",
  "AWS_DEFAULT_REGION": "us-east-2",
  "PWD": "/var/task",
  "AWS_SECRET_ACCESS_KEY": "Lrl9onnQsEw23uiottrTmf9V/AyDT5KqcTEV57vH",
  "LANG": "en_US.UTF-8",
  "LAMBDA_RUNTIME_DIR": "/var/runtime",
  "AWS_LAMBDA_INITIALIZATION_TYPE": "on-demand",
  "NODE_PATH": "/opt/nodejs/node18/node_modules:/opt/nodejs/node_modules:/var/runtime/node_modules:/var/runtime:/var/task",
  "TZ": ":UTC",
  "AWS_REGION": "us-east-2",
  "URL": "https://jss-nextjs-hosting.netlify.app",
  "AWS_ACCESS_KEY_ID": "ASIAVTMFWHR4Y3B6VRLM",
  "SHLVL": "0",
  "_AWS_XRAY_DAEMON_ADDRESS": "169.254.79.129",
  "_AWS_XRAY_DAEMON_PORT": "2000",
  "AWS_XRAY_CONTEXT_MISSING": "LOG_ERROR",
  "_HANDLER": "___netlify-handler.handler",
  "AWS_LAMBDA_FUNCTION_MEMORY_SIZE": "1024",
  "SITE_NAME": "jss-nextjs-hosting",
  "NODE_EXTRA_CA_CERTS": "/var/runtime/ca-cert.pem",
  "NODE_ENV": "production",
  "PUBLIC_URL": "https://jss-nextjs-hosting.netlify.app",
  "_X_AMZN_TRACE_ID": "Root=1-65d78c93-5c5400673898ab526c80658c;Parent=017ac75363729ae4;Sampled=0;Lineage=dd45a4d6:0",
  "__NEXT_PROCESSED_ENV": "true",
  "JSS_EDITING_SECRET": "...",
  "SITECORE_API_KEY": "",
  "SITECORE_API_HOST": "",
  "GRAPH_QL_ENDPOINT": "",
  "DEFAULT_LANGUAGE": "",
  "FETCH_WITH": "GraphQL",
  "DISABLE_SSG_FETCH": "",
  "__NEXT_OPTIMIZE_FONTS": "true",
  "__NEXT_PRIVATE_PREBUNDLED_REACT": ""
}

Does Next.js pages API route have ability to use Blobs?
It works with siteID and token. But is it expected that I need to configure them for runtime?
If yes, then it should be specified in the documentation.

No duplicates please @Antonytm: Environment variables inconsistency

For v4 of Next.js Runtime, yes for both questions.

I wasn’t able to find it in the documentation. I checked:
Netlify Blobs | Netlify Docs (Blobs)
next-runtime/README.md at main · netlify/next-runtime · GitHub (next runtime)

Can you point me out, where can I find it, please?

1 Like

Next Runtime v4 was not made in the time when Blobs existed. And when Blobs were made, work on Next Runtime v5 had already begun in which case updating the documentation for a niche use-case was not the highest priority. Most of what you need is documented here anyways: blobs/README.md at main · netlify/blobs (github.com)

Was this issue ever resolved for Svelte? Having the same issue here with SvelteKit. Inside my request I simply have:

const store = getStore('images');
await store.set('latest.jpg', imageBuffer);

When hitting the live endpoint on the deployed site, I get the following error:
ERROR MissingBlobsEnvironmentError: The environment has not been configured to use Netlify Blobs. To use it manually, supply the following properties when creating a store: siteID, token

It’s not yet resolved. Until SvelteKit changes its adapter to use the Functions API v2: kit/packages/adapter-netlify/src/serverless.js at main · sveltejs/kit (github.com), this won’t work.
An alternative would be to use the connectLambda function: netlify/blobs: A TypeScript client for Netlify Blobs (github.com) which can be called anywhere in the Lambda code as long as you can pass the event. The Svelte adapter provides you access to context but not event: kit/packages/adapter-netlify/src/serverless.js at main · sveltejs/kit (github.com).

A “hack” would be to add a postinstall script or something similar, that can modify the adapter’s code to add the connectLambda function in there. I’m not sure if adding this feature to the official adapter works or not, as it might or might not be needed by all.

Hi there …
I think there is something missing on the Netlify Environment !!
I was having the same issues as described here !

What worked for me:
I created a personal personal Access Token created the NETLIFY_BLOBS_CONTEXT environment variable myself and added that to my sites ENV vars !!

And then it worked → this basically concludes, that netlify IS NOT POPULATING the ENV var automatically as written here:
This data is automatically populated by Netlify in the execution environment for both serverless and edge functions. GitHub - netlify/blobs: A TypeScript client for Netlify Blobs

@hrishikesh :slight_smile:
Can you please check with your OPs team, why the ENV variable NETLIFY_BLOBS_CONTEXT is not set automatically?? Why do I have to generate that myself?
Thanks for any feedback

I have answered it above:

The only way it would work out of the box is if SvekteKit upgrades to Functions v2.

Another alternative would be to use the connectLambda function that I mentioned above at this line: kit/packages/adapter-netlify/src/serverless.js at main · sveltejs/kit (github.com) (insert it there).

Instead of mentioning about this issue here, if someone can point this to SvelteKit to make the necessary change, that could be more useful.

Hey @hrishikesh
I am sorry…I was a bit “tense” yesterday. I apologize for my rough tone in my writings…

Just to clear things up:
I should have raised a separate thread maybe, since my “findings” are not related to Svelte or SvelteKit.
I have a NuxtJS project.

But the problem was very similar to the one described here.

Let me give a small rundown, on my problem:

  • I created a simple Nuxt project and linked it to Netlify via cli
  • I started to use the Netlify Blobs on my local machine and it worked fine (-> I saw the local folder ./netlify/blobs-serve
    => then I pushed my project to Netlify and it did not work
  • I read the docs and a lot of other things and came across the thread
  • I tried a lot of things
    When I manually created the ENV var NETLIFY_BLOBS_CONTEXT in my Netlify site configuration,
    it started to work just fine !!
    So my conclusion was:

In the documentation it says
This data is automatically populated by Netlify in the execution environment for both serverless and edge functions.GitHub - netlify/blobs: A TypeScript client for Netlify Blobs

BUT it is not! or at least it was not in my project!
And I think @Antonytm mentioned it here as well => see comment with all ENV vars

So, I guess there is maybe a general issue, that the platform is missing the global ENV var NETLIFY_BLOBS_CONTEXT in new projects…

How should we proceed? Can you pass this on, or should I open a new Bug report / Formum thread?

Have a great day, and thanks a lot for your fast response!
(and again: Apologies for my last rough comment)
Tobi

Oh, since you raised the issue in a thread talking mainly about SvelteKit, I assumed you have the same issue. My bad.

This part is still true, but as mentioned above:

Only Functions API v2 automatically get that part configured. That’s because we add our own layer along that Function which adds the required data. In Functions v1, that’s currently not possible. Nuxt as well as SvelteKit still use Functions v1 and thus the issue. Next.js, Astro, Remix use Functions v2 and thus they are able to use this feature without an issue.

While writing this, I just checked and looks like Nitro (the server engine that powers Nuxt) is also being upgraded to Functions v2: nitro/src/presets/netlify/runtime/netlify.ts at main · unjs/nitro (github.com)

If that change has been shipped and has landed in Nuxt, you should not have to configure anything else and it should work out of the box.

The solutions:

  • Ask SvelteKit/Nuxt to upgrade to Functions v2 (based on my note above, Nuxt might have already been upgraded). For SvelteKit, I have provided an alternative way to use connectLambda() function.
  • You can switch to netlify_edge preset that would run on Edge Functions and you should be able to use Blobs there directly as well.
  • Use Netlify Blobs in API mode by setting the required env yourself as you’re already doing.

I’ve attempted to write an adapter for SvelteKit to be able to use Functions API v2, and in turn, Netlify Blobs as well. It seems to be working fine for me. I could submit a PR to SvelteKit repo, but it might not be merged any time soon. I could also release this as a separate NPM package, but before I do that, it would be great if others can let me know their experience. To opt it, add the following code to a JS file (call it adapter.js for example):

import { build } from 'esbuild';
import { builtinModules } from 'node:module';
import { cwd } from 'node:process';
import { join, relative } from 'node:path';
import { resolveConfig } from '@netlify/config';
import { writeFileSync } from 'node:fs';
const fn_name = 'SvelteKit Server';
const generator = '@netlify/adapter-sveltekit@5.0.0';
const working_dir = cwd();
const ntl_frameworks_api_dir = join(working_dir, './.netlify/v1/');
const sk_server_dir = join(working_dir, './.svelte-kit/netlify/');
async function init(builder) {
    const ntl_config = await resolveConfig({});
    const ntl_config_json = {
        build: {
            publish: ntl_config.config.build.publish
        },
        headers: [{
                for: `/${builder.getAppPath()}/immutable/*`,
                values: {
                    'cache-control': 'immutable, public, max-age=3153600'
                }
            }]
    };
    if (ntl_config_json.build.publish === working_dir || !ntl_config_json.build.publish) {
        ntl_config_json.build.publish = join(working_dir, './build/');
    }
    const sk_publish_path = join(ntl_config_json.build.publish, builder.config.kit.paths.base);
    builder.rimraf(ntl_config_json.build.publish);
    builder.rimraf(ntl_frameworks_api_dir);
    builder.rimraf(sk_server_dir);
    builder.mkdirp(ntl_frameworks_api_dir);
    builder.writeClient(sk_publish_path);
    builder.writePrerendered(sk_publish_path);
    builder.writeServer(sk_server_dir);
    writeFileSync(join(ntl_frameworks_api_dir, './config.json'), JSON.stringify(ntl_config_json));
}
export function adapterNetlifyEdgeFunctions(options = {}) {
    return {
        async adapt(builder) {
            await init(builder);
            const ntl_edge_functions_dir = join(ntl_frameworks_api_dir, './edge-functions/');
            const efn = `import {Server} from '${join(builder.getServerDirectory(), './index.js')}'
const server = new Server(${builder.generateManifest({
                relativePath: './'
            })})
await server.init({
  env: Deno.env.toObject()
})
export default async function(req, context) {
  return server.respond(req, {
    getClientAddress() {
      return context.ip
    },
    platform: {
      context
    }
  })
}
export const config = {
  excludedPath: ${JSON.stringify([
                '/.netlify/*',
                `/${builder.getAppPath()}/immutable/*`
            ].concat(builder.prerendered.paths).concat(options.excludedPath || []))},
  generator: '${generator}',
  name: '${fn_name}',
  onError: ${JSON.stringify(options.onError || undefined)},
  path: '/*',
  rateLimit: ${JSON.stringify(options.rateLimit || {})}
}`;
            builder.mkdirp(ntl_edge_functions_dir);
            writeFileSync(join(sk_server_dir, './sk-server.js'), efn);
            await build({
                alias: Object.fromEntries(builtinModules.map(id => [
                    id,
                    `node:${id}`
                ])),
                bundle: true,
                entryPoints: [
                    join(sk_server_dir, './sk-server.js')
                ],
                external: builtinModules.map(id => `node:${id}`),
                format: 'esm',
                outfile: join(ntl_edge_functions_dir, './sk-server.js'),
                platform: 'browser',
                target: 'esnext'
            });
        },
        name: generator,
        supports: {
            read(config) {
                throw new Error(`${generator} cannot use \`read\` from \`$apps/server\' in route \`${config.route.id}\`, switch to \`adapterNetlifyFunctions\``);
            }
        }
    };
}
export function adapterNetlifyFunctions(options = {}) {
    return {
        async adapt(builder) {
            await init(builder);
            const ntl_functions_dir = join(ntl_frameworks_api_dir, './functions/');
            const fn = `import {env} from 'node:process'
import {File} from 'node:buffer'
import {Server} from '${join(builder.getServerDirectory(), './index.js')}'
import {webcrypto} from 'node:crypto'
for (const name in {
  crypto: webcrypto,
  File
}) {
  if (name in globalThis) {
    continue
  }
  Object.defineProperty(globalThis, name, {
    configurable: true,
    enumerable: true,
    value: globals[name],
    writable: true
  })
}
const server = new Server(${builder.generateManifest({
                relativePath: relative(ntl_functions_dir, sk_server_dir)
            })})
await server.init({
  env
})
export default async function(req, context) {
  return server.respond(req, {
    getClientAddress() {
      return context.ip
    },
    platform: {
      context
    }
  })
}
export const config = {
  displayName: '${fn_name}',
  excludedPath: ${JSON.stringify([
                '/.netlify/*'
            ].concat(options.excludedPath || []))},
  generator: '${generator}',
  path: '/*',
  preferStatic: true,
  rateLimit: ${JSON.stringify(options.rateLimit || {})}
}`;
            builder.mkdirp(ntl_functions_dir);
            writeFileSync(join(ntl_functions_dir, './sk-server.js'), fn);
        },
        name: generator
    };
}

In your svelte.config.js:

- import adapter from '@sveltejs/adapter-netlify'
+ import {adapterNetlifyFunctions} from './adapter.js' // change the path to the adapter.js file
const config = {
  ...
  kit: {
    adapter: adapterNetlifyFunctions({}),
  }
  ...
}

Also, install the following dependencies:

esbuild
@netlify/config

Feel free to let me know if someone runs into any issues due to this adapter. The feedback will help to test and release this.