Response text is streaming only as a single chunk

Support
, ,
1

We’re trying to build a simple OpenAI chat widget, but are having troubles with text response streaming.

new Response(new ReadableStream(...))

I’ve created a simple reproduction app with SvelteKit

https://text-streaming-test.vercel.app - works great on Vercel, the text is streamed in chunks and progressively rendered

https://text-stream.netlify.app - doesn’t work as expected on Netlify, text is returned as a single chunk, so it seems like it waits until the ReadableStream is complete before sending the response

Both are on free tier, but I don’t think the timeout is the issue as the chunks start streaming prior to the limit.
The code is identical apart from the build adapters.

Is this a limitation of the platform?

2

SvelteKit still uses Netlify Functions v1: kit/packages/adapter-netlify/src/serverless.js at main · sveltejs/kit. It requires extra steps for streaming: Lambda compatibility for Functions | Netlify Docs, but SvelteKit does nothing to implement that conversion in its adapter. Functions v2 can natively stream data: Get started with functions | Netlify Docs, quoting:

Synchronous functions can stream data to clients as it becomes available, rather than returning a buffered payload at the end of the computation. This lets developers and frameworks create faster experiences by using streaming and partial hydration to get content and interactions in front of people as quickly as possible.

To stream a function’s response, return a ReadableStream as the body property of the Response object.

But SvelteKit is yet to upgrade itself to use that. I had attempted to write an adapter to use Functions v2: Netlify Blobs: accessing from function (inside SvelteKit app) requiring explicit token, siteId, deployId - #21 by hrishikesh. You can try using that if you’d like (though I haven’t extensively tested it - it worked in a basic hello-world kinda app).