Streaming AI Responses in Next.js with the Vercel AI SDK

You have built the agent. You wired up the RAG pipeline. You connected the function calling loop. Then you show it to someone — and they stare at a blank screen for four seconds before the full response appears at once.

That is the moment you realize streaming is not a nice-to-have. It is the difference between something that feels intelligent and something that feels like a slow API call.

The Vercel AI SDK solves this in Next.js with remarkably little code. In this guide you will implement production-ready AI streaming — from a simple text stream all the way to streaming tool calls and structured objects — using Next.js App Router and the Vercel AI SDK.

Why Streaming Matters for AI UX

A non-streamed AI response forces the user to wait in silence. A streamed response starts showing text within 300–500ms and builds naturally — the same way a person types. The perceived wait drops dramatically even when the total generation time is identical.

Three concrete reasons streaming belongs in every AI product:

• Time to first token beats total latency. Users tolerate 5 seconds of streaming far better than 2 seconds of blank waiting. The interaction feels alive.
• Users can bail early. If the model starts going in the wrong direction, a user can stop it and rephrase. Without streaming they wait for the full wrong answer before they can correct course.
• It signals quality. A streaming response looks like thinking. A delayed bulk response looks like a slow database query. The perception of intelligence is directly tied to the visual pattern.

Users forgive slow generation. They do not forgive a blank screen.

How Streaming Works in Next.js — The Architecture

BrowseruseChat hook

user hits send

Next.js route handlerPOST /api/chat

{ messages }

Vercel AI SDKstreamText()

opens the OpenAI stream

OpenAIgpt-4o · streaming

token chunks stream back

~300ms to first token

Vercel AI SDKtoDataStreamResponse()

ReadableStream over HTTP

Browsertokens render live

useChat re-renders per token

The Vercel AI SDK sits between your Next.js route handler and OpenAI. It manages the streaming connection on the server side and exposes a clean ReadableStream to the browser. The useChat hook on the client consumes that stream and updates React state token by token — no manual EventSource or WebSocket setup needed.

Project Setup

If you followed the previous articles in this series — the agent build, the RAG pipeline — you already have a Next.js project. Add the Vercel AI SDK:

pnpm add ai @ai-sdk/openai

That is it. The ai package is the core SDK; @ai-sdk/openai is the OpenAI provider adapter. Add your key to .env.local:

OPENAI_API_KEY=sk-your-key-here

Part 1 — Server Side: The Streaming Route Handler

Create app/api/chat/route.ts:

import { openai } from "@ai-sdk/openai";
import { streamText } from "ai";
import { NextRequest } from "next/server";

// Required for streaming on Vercel — sets max function duration
export const maxDuration = 30;

export async function POST(req: NextRequest) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai("gpt-4o"),
    system: `You are a helpful AI assistant.
Be concise and clear in your responses.`,
    messages,
  });

  // toDataStreamResponse() converts the stream into the format
  // the useChat hook expects on the client
  return result.toDataStreamResponse();
}

That is the complete streaming server. streamText opens a streaming connection to OpenAI, and toDataStreamResponse() pipes it back to the browser in the SDK's data stream protocol.

Why export const maxDuration = 30? Vercel serverless functions default to a 10-second timeout. AI responses can easily exceed that. Set maxDuration to 30 on Pro plans — or 60 if you expect long responses.

Part 2 — Client Side: The useChat Hook

Create app/page.tsx:

"use client";

import { useChat } from "ai/react";
import { useEffect, useRef } from "react";

export default function ChatPage() {
  const {
    messages,
    input,
    handleInputChange,
    handleSubmit,
    isLoading,
    stop,    // Cancel an in-progress stream
    error,   // Any stream error
  } = useChat({
    api: "/api/chat",
    onError: (error) => {
      console.error("Stream error:", error);
    },
  });

  // Auto-scroll to the latest message
  const bottomRef = useRef<HTMLDivElement>(null);
  useEffect(() => {
    bottomRef.current?.scrollIntoView({ behavior: "smooth" });
  }, [messages]);

  return (
    <div className="flex flex-col h-screen max-w-2xl mx-auto p-4">

      {/* Message thread */}
      <div className="flex-1 overflow-y-auto space-y-4 py-4">
        {messages.map((message) => (
          <div
            key={message.id}
            className={`flex ${
              message.role === "user" ? "justify-end" : "justify-start"
            }`}
          >
            <div
              className={`max-w-[85%] rounded-2xl px-4 py-2 text-sm ${
                message.role === "user"
                  ? "bg-blue-600 text-white"
                  : "bg-gray-100 text-gray-900"
              }`}
            >
              {/*
                message.content updates token by token during streaming.
                React re-renders on each update — no extra code needed.
              */}
              {message.content}
            </div>
          </div>
        ))}

        {/* Loading indicator — shows while waiting for first token */}
        {isLoading && (
          <div className="flex justify-start">
            <div className="bg-gray-100 rounded-2xl px-4 py-2">
              <span className="text-sm text-gray-400 animate-pulse">
                Thinking...
              </span>
            </div>
          </div>
        )}

        {/* Error state */}
        {error && (
          <div className="text-center text-sm text-red-500">
            Something went wrong. Please try again.
          </div>
        )}

        <div ref={bottomRef} />
      </div>

      {/* Input area */}
      <form onSubmit={handleSubmit} className="flex gap-2 pt-4 border-t">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask anything..."
          className="flex-1 border rounded-xl px-4 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-blue-500"
          disabled={isLoading}
        />

        {/* Show Stop button while streaming, Send button otherwise */}
        {isLoading ? (
          <button
            type="button"
            onClick={stop}
            className="bg-red-500 text-white px-4 py-2 rounded-xl text-sm hover:bg-red-600 transition"
          >
            Stop
          </button>
        ) : (
          <button
            type="submit"
            disabled={!input.trim()}
            className="bg-blue-600 text-white px-4 py-2 rounded-xl text-sm disabled:opacity-40 hover:bg-blue-700 transition"
          >
            Send
          </button>
        )}
      </form>
    </div>
  );
}

The useChat hook does the heavy lifting — it manages message state, handles the streaming connection, and exposes isLoading and stop for UI control. message.content updates token by token as the stream arrives, and React re-renders automatically.

Part 3 — Streaming Tool Calls

This is where the Vercel AI SDK really separates itself. Streaming tool calls with the raw OpenAI SDK means manually assembling argument chunks as they arrive. The Vercel AI SDK handles it automatically.

streamText()gpt-4o · tools · maxSteps: 3

text delta

stream to browserinstantly · token by token

tool call

execute()runs on the server

tool result → modelstream continues

browseruseChat renders text + tool results live

Update your route handler to include tools:

import { openai } from "@ai-sdk/openai";
import { streamText, tool } from "ai";
import { z } from "zod";
import { NextRequest } from "next/server";

export const maxDuration = 30;

export async function POST(req: NextRequest) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai("gpt-4o"),
    system: "You are a helpful assistant with access to weather data.",
    messages,
    tools: {
      // The Vercel AI SDK uses a cleaner tool definition syntax
      getWeather: tool({
        description:
          "Get the current weather for a city. Use when the user asks about weather.",
        parameters: z.object({
          city: z.string().describe("The city name"),
          unit: z
            .enum(["celsius", "fahrenheit"])
            .describe("Temperature unit"),
        }),
        // execute runs on the server — result streams back to the client automatically
        execute: async ({ city, unit }) => {
          // Replace with a real weather API call
          return {
            city,
            temperature: unit === "celsius" ? 18 : 64,
            condition: "Partly cloudy",
            unit,
          };
        },
      }),
    },
    maxSteps: 3, // Allow up to 3 tool call + response cycles
  });

  return result.toDataStreamResponse();
}

maxSteps: 3 is the key parameter. Without it, the model calls the tool and stops — it does not automatically incorporate the result and generate a final response. With maxSteps, the SDK runs the full loop: call tool → get result → generate response.

Without maxSteps the model calls the tool and stops. With it, the whole loop happens inside one streaming connection.

The client code stays identical — useChat handles tool call streaming transparently. Your UI does not need to change.

Part 4 — Streaming Structured Objects

Sometimes you do not want a text stream — you want a streaming JSON object. The AI SDK's streamObject is built for this:

// app/api/analyze/route.ts
import { openai } from "@ai-sdk/openai";
import { streamObject } from "ai";
import { z } from "zod";
import { NextRequest } from "next/server";

export const maxDuration = 30;

// Define the shape of the object you want streamed
const analysisSchema = z.object({
  summary: z.string().describe("One sentence summary"),
  sentiment: z.enum(["positive", "negative", "neutral"]),
  keyPoints: z.array(z.string()).describe("Main points, max 5"),
  score: z.number().min(0).max(10).describe("Quality score"),
});

export async function POST(req: NextRequest) {
  const { text } = await req.json();

  const result = streamObject({
    model: openai("gpt-4o"),
    schema: analysisSchema,
    prompt: `Analyze the following text:\n\n${text}`,
  });

  return result.toTextStreamResponse();
}

On the client, use useObject to consume it:

"use client";

import { experimental_useObject as useObject } from "ai/react";
import { z } from "zod";

const analysisSchema = z.object({
  summary: z.string(),
  sentiment: z.enum(["positive", "negative", "neutral"]),
  keyPoints: z.array(z.string()),
  score: z.number(),
});

export default function AnalyzePage() {
  const { object, submit, isLoading } = useObject({
    api: "/api/analyze",
    schema: analysisSchema,
  });

  return (
    <div className="max-w-xl mx-auto p-6">
      <button
        onClick={() => submit({ text: "Your text to analyze here..." })}
        className="bg-blue-600 text-white px-4 py-2 rounded-lg"
        disabled={isLoading}
      >
        {isLoading ? "Analyzing..." : "Analyze"}
      </button>

      {/* object properties appear and update as they stream in */}
      {object && (
        <div className="mt-6 space-y-3">
          {object.summary && <p>{object.summary}</p>}
          {object.sentiment && (
            <span className="badge">{object.sentiment}</span>
          )}
          {object.keyPoints?.map((point, i) => (
            <li key={i}>{point}</li>
          ))}
          {object.score !== undefined && (
            <p>Score: {object.score}/10</p>
          )}
        </div>
      )}
    </div>
  );
}

The object properties stream in and render as they arrive — summary might appear before keyPoints finishes generating. This creates a compelling progressive UI with zero extra complexity.

Part 5 — Cancelling Streams Mid-Generation

The stop() function from useChat sends an abort signal to the server, which cancels the OpenAI connection immediately. No tokens are wasted after the user clicks stop.

const { stop, isLoading } = useChat();

// In your JSX:
{isLoading && (
  <button onClick={stop} className="...">
    ⏹ Stop generating
  </button>
)}

On the server side, streamText handles the abort signal automatically — no extra code needed.

Part 6 — Error Handling in Production Streams

Streams fail differently from regular API calls. The connection is already open when the error occurs, so you cannot just return a 500 status. Handle errors at two levels.

Server level — catch before streaming starts:

export async function POST(req: NextRequest) {
  try {
    const { messages } = await req.json();

    // Validate input before opening the stream
    if (!messages || !Array.isArray(messages)) {
      return new Response("Invalid request", { status: 400 });
    }

    const result = streamText({
      model: openai("gpt-4o"),
      messages,
      onError: (error) => {
        // Log mid-stream errors (e.g. OpenAI rate limits)
        console.error("Stream error:", error);
      },
    });

    return result.toDataStreamResponse();
  } catch (error) {
    // Catches errors before the stream opens (auth, validation, etc.)
    return new Response("Failed to start stream", { status: 500 });
  }
}

Client level — handle gracefully in the UI:

const { error, reload } = useChat({
  onError: (error) => {
    console.error("Client stream error:", error);
    // Show toast notification, log to Sentry, etc.
  },
});

// Show a retry option when the stream fails
{error && (
  <div className="flex items-center gap-2 text-sm text-red-500">
    <span>Something went wrong.</span>
    <button onClick={reload} className="underline">
      Try again
    </button>
  </div>
)}

reload() from useChat retries the last message automatically — no need to re-type.

Adding Rate Limiting Before You Go Live

Never deploy a streaming AI endpoint without rate limiting. A single automated client can drain your OpenAI budget in minutes.

pnpm add @upstash/ratelimit @upstash/redis

// app/api/chat/route.ts — add rate limiting
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";

const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(20, "1 m"), // 20 requests/min per user
});

export async function POST(req: NextRequest) {
  // Use IP as identifier (use user ID if you have auth)
  const ip = req.headers.get("x-forwarded-for") ?? "anonymous";
  const { success } = await ratelimit.limit(ip);

  if (!success) {
    return new Response("Too many requests", { status: 429 });
  }

  // ... rest of your handler
}

Deployment on Vercel

Push to GitHub and deploy. Two things to configure in your Vercel dashboard:

1. Environment variable:

OPENAI_API_KEY=sk-...

2. Function duration — already handled by export const maxDuration = 30 in your route file.

user browseranywhere on earth

vercel edge networkroutes to nearest region

next.js serverless functionnodejs · maxDuration: 30

openai apigpt-4o · streaming

~300ms TTFT tokens · ReadableStream

For streaming specifically, Vercel's infrastructure is the natural choice — native support for streaming responses, and Edge Runtime if you want to shave cold-start latency off the first token.

Quick Reference — Which Hook for Which Use Case

Frequently asked questions

Where to go next

You now have the complete streaming foundation — everything earlier in this series drops straight into it:

• How to Build an AI Agent with LangChain and OpenAI Combine streaming with the agent loop for a streaming agentic experience. →
• Building a RAG Application with Next.js, OpenAI, and Pinecone Make the chat endpoint retrieve from Pinecone before it streams. →
• OpenAI Function Calling for Node.js Developers Those tool definitions drop directly into the AI SDK's tools parameter. →

Text, tool calls, structured objects, cancellation, errors, rate limits — that is the whole streaming foundation. Bolt it onto your agent, your RAG pipeline, or your function calling loop, and four seconds of silence becomes the first 300 milliseconds of an answer.