Streaming State Machine

The Boolean Graveyard

Here's some code that ships to production every day:

const [isLoading, setIsLoading] = useState(false);
const [isStreaming, setIsStreaming] = useState(false);
const [error, setError] = useState<string | null>(null);
const [isConnecting, setIsConnecting] = useState(false);
const [isCancelled, setIsCancelled] = useState(false);

Five booleans. Each one true or false. That's 2^5 = 32 possible combinations. How many of those are valid? Maybe 6. The other 26 are bugs waiting to happen -- isLoading is true but isStreaming is also true and error is set and isCancelled is true. What does the UI show? Nobody knows. The component doesn't know either.

This is the problem that kills AI chat interfaces. Not the streaming protocol, not the API integration -- the state management. And the fix is something computer science figured out decades ago: a finite state machine.

The Mental Model

Why Booleans Break Streaming UIs

Let's trace a real bug. Your user clicks "Send", and you set isLoading = true. The stream starts, so you set isStreaming = true and isLoading = false. Tokens arrive. Then the user clicks "Stop Generating". You set isCancelled = true and isStreaming = false. But wait -- an error event fires from the aborted connection. Now error gets set too. Your UI shows an error message over the cancelled state. The user sees "Something went wrong" when they intentionally stopped the response.

Here's why this happens:

// The boolean approach — each setter is independent
async function handleSend(message: string) {
  setError(null);
  setIsLoading(true);
  setIsConnecting(true);

  try {
    const stream = await startStream(message);
    setIsConnecting(false);
    setIsStreaming(true);
    setIsLoading(false);

    for await (const chunk of stream) {
      // What if the user cancelled during this loop?
      // isCancelled might be true but isStreaming is also true
      appendToken(chunk);
    }

    setIsStreaming(false);
  } catch (err) {
    // Was this error from cancellation or a real failure?
    // We can't tell — isCancelled might or might not be true
    setError(err.message);
    setIsLoading(false);
    setIsStreaming(false);
    setIsConnecting(false);
  }
}

Every set* call is independent. Nothing prevents you from reaching a contradictory state. And in async code with race conditions, you will reach one.

The Six States of Streaming

Every LLM streaming interaction follows this lifecycle:

Valid Transitions

Not every state can reach every other state. This is the whole point of a state machine -- invalid transitions are impossible by design:

Notice what's not in this table. You can't go from Idle to Streaming (you must connect first). You can't go from Error to Streaming (you must retry, which goes through Connecting). You can't go from Cancelling to Error (cancellation is intentional, not an error).

Mapping States to UI

Each state maps to exactly one UI configuration. No ambiguity, no conditional soup:

function getUIForState(state: StreamState) {
  switch (state.status) {
    case 'idle':
      return {
        showInput: true,
        showSpinner: false,
        showStopButton: false,
        showRetryButton: false,
        inputDisabled: false,
        placeholder: 'Type a message...',
      };
    case 'connecting':
      return {
        showInput: true,
        showSpinner: true,
        showStopButton: true,
        showRetryButton: false,
        inputDisabled: true,
        placeholder: 'Connecting...',
      };
    case 'streaming':
      return {
        showInput: true,
        showSpinner: false,
        showStopButton: true,
        showRetryButton: false,
        inputDisabled: true,
        placeholder: 'Generating...',
      };
    case 'cancelling':
      return {
        showInput: true,
        showSpinner: true,
        showStopButton: false,
        showRetryButton: false,
        inputDisabled: true,
        placeholder: 'Stopping...',
      };
    case 'error':
      return {
        showInput: true,
        showSpinner: false,
        showStopButton: false,
        showRetryButton: true,
        inputDisabled: false,
        placeholder: 'Type a message or retry...',
      };
    case 'complete':
      return {
        showInput: true,
        showSpinner: false,
        showStopButton: false,
        showRetryButton: false,
        inputDisabled: false,
        placeholder: 'Type a message...',
      };
  }
}

Compare this to the boolean approach:

// Boolean hell — every render has to untangle combinations
const showSpinner = isLoading || isConnecting;
const showStopButton = isStreaming && !isCancelled;
const showRetryButton = !!error && !isLoading;
const inputDisabled = isLoading || isStreaming || isConnecting;
// What about isStreaming && isCancelled? isLoading && error?
// Every new feature adds more boolean spaghetti.

Handling User Actions Per State

Here's where the state machine really shines. Each state defines which user actions are valid:

function canSend(status: StreamStatus): boolean {
  return status === 'idle' || status === 'complete' || status === 'error';
}

function canCancel(status: StreamStatus): boolean {
  return status === 'connecting' || status === 'streaming';
}

function canRetry(status: StreamStatus): boolean {
  return status === 'error';
}

With booleans, you'd check !isLoading && !isStreaming && !isConnecting. Miss one? Bug. Add a new flag? Check every condition. With a state machine, you check one value against an explicit list.

Race Condition Prevention

The nastiest bug in streaming UIs: the user sends message A, then quickly sends message B before A finishes streaming. Now you have two active streams writing to the same message array. Tokens interleave. The response is garbled.

A state machine prevents this structurally:

case 'SEND': {
  // Only allow sending from idle, complete, or error
  if (
    state.status !== 'idle' &&
    state.status !== 'complete' &&
    state.status !== 'error'
  ) {
    return state; // Reject — we're connecting, streaming, or cancelling
  }

  return {
    status: 'connecting',
    messages: [
      ...state.messages,
      { role: 'user', content: action.message },
    ],
    currentResponse: '',
    error: null,
    abortController: new AbortController(),
  };
}

The second send attempt arrives while status is 'connecting' or 'streaming'. The reducer returns the current state unchanged. No second request fires. No interleaving. No race condition.

But what if the user wants to send a new message? They need to cancel first. The state machine enforces the flow: Streaming to Cancelling to Idle, then they can send again.

The AbortController Pattern

Cancellation needs to work at both the state level and the network level:

case 'CANCEL': {
  if (
    state.status !== 'connecting' &&
    state.status !== 'streaming'
  ) {
    return state;
  }

  return {
    ...state,
    status: 'cancelling',
  };
}

case 'CANCEL_COMPLETE': {
  if (state.status !== 'cancelling') {
    return state;
  }

  return {
    ...state,
    status: 'idle',
    abortController: null,
  };
}

The CANCEL action transitions state to cancelling. A separate useEffect watches for this state and calls abort() on the controller -- keeping the side effect out of the reducer. The CANCEL_COMPLETE action fires after cleanup finishes. This two-step process prevents the abort error from being treated as a real error -- by the time the error callback fires, the state is 'cancelling', and the reducer ignores error events in that state:

case 'ERROR': {
  // Ignore errors during cancellation — they're expected
  if (state.status === 'cancelling') {
    return state;
  }

  if (
    state.status !== 'connecting' &&
    state.status !== 'streaming'
  ) {
    return state;
  }

  return {
    ...state,
    status: 'error',
    error: action.error,
    abortController: null,
  };
}

This is the fix for the "error after cancel" bug we described at the beginning. The state machine makes it a non-issue.

Implementation with useReducer

Here's the complete implementation. This is the clean pattern:

Types

type StreamStatus =
  | 'idle'
  | 'connecting'
  | 'streaming'
  | 'cancelling'
  | 'error'
  | 'complete';

type Message = {
  role: 'user' | 'assistant';
  content: string;
};

type StreamState = {
  status: StreamStatus;
  messages: Message[];
  currentResponse: string;
  error: string | null;
  abortController: AbortController | null;
};

type StreamAction =
  | { type: 'SEND'; message: string }
  | { type: 'CONNECTED' }
  | { type: 'TOKEN'; token: string }
  | { type: 'COMPLETE' }
  | { type: 'CANCEL' }
  | { type: 'CANCEL_COMPLETE' }
  | { type: 'ERROR'; error: string }
  | { type: 'RETRY' };

Notice the discriminated union on StreamAction. TypeScript narrows the action type inside each case, so you get full type safety on payload access. No action.message on a TOKEN action -- the compiler catches it.

The Reducer

const initialState: StreamState = {
  status: 'idle',
  messages: [],
  currentResponse: '',
  error: null,
  abortController: null,
};

function streamReducer(
  state: StreamState,
  action: StreamAction
): StreamState {
  switch (action.type) {
    case 'SEND': {
      if (
        state.status !== 'idle' &&
        state.status !== 'complete' &&
        state.status !== 'error'
      ) {
        return state;
      }

      return {
        status: 'connecting',
        messages: [
          ...state.messages,
          { role: 'user', content: action.message },
        ],
        currentResponse: '',
        error: null,
        abortController: new AbortController(),
      };
    }

    case 'CONNECTED': {
      if (state.status !== 'connecting') return state;
      return { ...state, status: 'streaming' };
    }

    case 'TOKEN': {
      if (state.status !== 'streaming') return state;
      return {
        ...state,
        currentResponse: state.currentResponse + action.token,
      };
    }

    case 'COMPLETE': {
      if (state.status !== 'streaming') return state;
      return {
        ...state,
        status: 'complete',
        messages: [
          ...state.messages,
          { role: 'assistant', content: state.currentResponse },
        ],
        currentResponse: '',
        abortController: null,
      };
    }

    case 'CANCEL': {
      if (
        state.status !== 'connecting' &&
        state.status !== 'streaming'
      ) {
        return state;
      }
      return { ...state, status: 'cancelling' };
    }

    case 'CANCEL_COMPLETE': {
      if (state.status !== 'cancelling') return state;
      const partialResponse = state.currentResponse;
      return {
        ...state,
        status: 'idle',
        messages: partialResponse
          ? [
              ...state.messages,
              {
                role: 'assistant',
                content: partialResponse + ' [stopped]',
              },
            ]
          : state.messages,
        currentResponse: '',
        abortController: null,
      };
    }

    case 'ERROR': {
      if (state.status === 'cancelling') return state;
      if (
        state.status !== 'connecting' &&
        state.status !== 'streaming'
      ) {
        return state;
      }
      return {
        ...state,
        status: 'error',
        error: action.error,
        abortController: null,
      };
    }

    case 'RETRY': {
      if (state.status !== 'error') return state;
      const lastUserMessage = [...state.messages]
        .reverse()
        .find((m) => m.role === 'user');
      if (!lastUserMessage) return { ...state, status: 'idle' };

      return {
        ...state,
        status: 'connecting',
        error: null,
        abortController: new AbortController(),
      };
    }
  }
}

The Hook

function useStreamChat() {
  const [state, dispatch] = useReducer(streamReducer, initialState);

  const send = useCallback(
    async (message: string) => {
      dispatch({ type: 'SEND', message });
    },
    []
  );

  const cancel = useCallback(() => {
    dispatch({ type: 'CANCEL' });
  }, []);

  const retry = useCallback(() => {
    dispatch({ type: 'RETRY' });
  }, []);

  useEffect(() => {
    if (state.status !== 'cancelling') return;
    state.abortController?.abort();
  }, [state.status, state.abortController]);

  useEffect(() => {
    if (state.status !== 'connecting') return;

    const controller = state.abortController;
    if (!controller) return;

    let cancelled = false;

    (async () => {
      try {
        const response = await fetch('/api/chat', {
          method: 'POST',
          body: JSON.stringify({
            messages: state.messages,
          }),
          signal: controller.signal,
        });

        if (!response.ok) {
          dispatch({
            type: 'ERROR',
            error: `Server error: ${response.status}`,
          });
          return;
        }

        dispatch({ type: 'CONNECTED' });

        const reader = response.body?.getReader();
        const decoder = new TextDecoder();

        if (!reader) {
          dispatch({
            type: 'ERROR',
            error: 'No response body',
          });
          return;
        }

        while (true) {
          const { done, value } = await reader.read();
          if (done) break;

          const text = decoder.decode(value, { stream: true });
          dispatch({ type: 'TOKEN', token: text });
        }

        dispatch({ type: 'COMPLETE' });
      } catch (err) {
        if (cancelled) return;

        if (err instanceof DOMException && err.name === 'AbortError') {
          dispatch({ type: 'CANCEL_COMPLETE' });
        } else {
          dispatch({
            type: 'ERROR',
            error:
              err instanceof Error
                ? err.message
                : 'Unknown error',
          });
        }
      }
    })();

    return () => {
      cancelled = true;
    };
  }, [state.status, state.abortController, state.messages]);

  return {
    ...state,
    send,
    cancel,
    retry,
    canSend:
      state.status === 'idle' ||
      state.status === 'complete' ||
      state.status === 'error',
    canCancel:
      state.status === 'connecting' ||
      state.status === 'streaming',
    canRetry: state.status === 'error',
  };
}

The useEffect watches for the connecting status. When the reducer transitions to connecting, the effect fires and starts the fetch. This keeps the side effect out of the reducer (reducers must be pure) and out of the event handler (which would bypass the state machine).

How Vercel AI SDK Handles This

If you're using the Vercel AI SDK, you might wonder why you'd bother with a custom state machine. Here's the thing -- useChat already uses one internally. Understanding what it does helps you extend it:

import { useChat } from 'ai/react';

function Chat() {
  const {
    messages,
    input,
    handleInputChange,
    handleSubmit,
    isLoading,
    error,
    stop,
    reload,
    status,
  } = useChat();

  // status is: 'awaiting_message' | 'in_progress' | 'streaming' | 'error'
  // This is their state machine — fewer states than ours,
  // but the same idea
}

The SDK's status field maps to a simplified version of our model:

The SDK doesn't expose a cancelling state -- stop() immediately transitions back to awaiting_message. For many apps this is fine. But if you need to show "Stopping..." UI, preserve partial responses with metadata, or handle complex multi-turn cancellation, you'll want the full state machine.

Anthropic's Multi-Level State

Anthropic's streaming API adds another layer of complexity. A single message can contain multiple content blocks (text, tool use, thinking), and each block has its own lifecycle:

Message Stream:
  message_start          → message-level connecting
  content_block_start    → block-level connecting
  content_block_delta    → block-level streaming (tokens)
  content_block_stop     → block-level complete
  content_block_start    → another block starts
  content_block_delta    → more tokens
  content_block_stop     → block complete
  message_delta          → message-level metadata (stop_reason, usage)
  message_stop           → message-level complete

For most chat UIs, you can flatten this into our six-state model. But if you're building a tool-use agent where the AI calls functions mid-response, you might need nested state machines -- one for the overall message, one for the current content block. That's beyond our scope here, but the pattern is the same: explicit states, valid transitions, one state at a time.

Execution Trace: A Complete Flow

Let's trace what happens when a user sends a message, receives a partial response, and cancels:

Common Mistakes

Key Rules

Where to Go From Here