Error: Maximum update depth exceeded. When using useCompletion hook in nextjs, on long response on gpt-4o

[Jerry-VW]

Description

Use useCompletion from AI SDK to call gpt-4o that will have a long response in streaming mode.
It will hang the UI. Looks like its updating completion's state in a very fast pace.

Unhandled Runtime Error
Error: Maximum update depth exceeded. This can happen when a component repeatedly calls setState inside componentWillUpdate or componentDidUpdate. React limits the number of nested updates to prevent infinite loops.

Call Stack
throwIfInfiniteUpdateLoopDetected
node_modules/next/dist/compiled/react-dom/cjs/react-dom.development.js (26607:0)
getRootForUpdatedFiber
node_modules/next/dist/compiled/react-dom/cjs/react-dom.development.js (7667:0)
enqueueConcurrentRenderForLane
node_modules/next/dist/compiled/react-dom/cjs/react-dom.development.js (7589:0)
forceStoreRerender
node_modules/next/dist/compiled/react-dom/cjs/react-dom.development.js (11893:0)
handleStoreChange
node_modules/next/dist/compiled/react-dom/cjs/react-dom.development.js (11872:0)
eval
node_modules/ai/node_modules/swr/core/dist/index.mjs (141:0)
Array.setter
node_modules/ai/node_modules/swr/_internal/dist/index.mjs (408:0)
eval
node_modules/ai/node_modules/swr/_internal/dist/index.mjs (99:0)
mutateByKey
node_modules/ai/node_modules/swr/_internal/dist/index.mjs (353:0)
internalMutate
node_modules/ai/node_modules/swr/_internal/dist/index.mjs (261:0)
eval
node_modules/ai/node_modules/swr/core/dist/index.mjs (367:29)
setCompletion
node_modules/ai/react/dist/index.mjs (1057:0)
callCompletionApi
node_modules/ai/react/dist/index.mjs (970:0)

Code example

No response

Additional context

No response

[rossanodr]

Same here

[rossanodr]

---

Hey, I don't know if you've already solved the problem, but I managed to fix it, and maybe it can help you.

My issue was that I was using message.content in a map.

messages.map((m, i) => (
    <ChatMessage
        content={m.content}
    />
))

This was causing multiple updates.

I solved the problem by passing messages directly to the final component:

<ChatMessage content={messages} />

I hope this helps you.

[ElectricCodeGuy]

I also experience this issue. I only have it using the route hadler. After changing to the new rsc/ai i have not seen it. It must have something to do with the re-renders when streaming.

Sure! Here’s the translation to English:

Hey, I don't know if you've already solved the problem, but I managed to fix it, and maybe it can help you.

My issue was that I was using message.content in a map.

messages.map((m, i) => (
    <ChatMessage
        content={m.content}
    />
))

This was causing multiple updates.

I solved the problem by passing messages directly to the final component:

<ChatMessage content={messages} />

I hope this helps you.

So after reading this message here i think i finely solved the issue... Spend so much time on it xD

So before i had the chat message displayed like this here:

const ChatMessage: FC<ChatMessageProps> = ({ messages }) => {
  const [isCopied, setIsCopied] = useState(false);
  const router = useRouter();
  const componentsAI: Partial<Components> = {
    a: ({ href, children }) => (
      <a
        href={href}
        onClick={(e) => {
          e.preventDefault();
          if (href) {
            router.push(href);
          }
        }}
      >
        {children}
      </a>
    ),
    code({ className, children, ...props }) {
      const match = /language-(\w+)/.exec(className || '');
      const language = match && match[1] ? match[1] : '';
      const inline = !language;
      if (inline) {
        return (
          <code className={className} {...props}>
            {children}
          </code>
        );
      }

      return (
        <div
          style={{
            position: 'relative',
            borderRadius: '5px',
            padding: '20px',
            marginTop: '20px',
            maxWidth: '100%'
          }}
        >
          <span
            style={{
              position: 'absolute',
              top: '0',
              left: '5px',
              fontSize: '0.8em',
              textTransform: 'uppercase'
            }}
          >
            {language}
          </span>
          <div
            style={{
              overflowX: 'auto',
              maxWidth: '1100px'
            }}
          >
            <pre style={{ margin: '0' }}>
              <code className={className} {...props}>
                {children}
              </code>
            </pre>
          </div>
        </div>
      );
    }
  };

  const componentsUser: Partial<Components> = {
    a: ({ href, children }) => (
      <a
        href={href}
        onClick={(e) => {
          e.preventDefault();
          if (href) {
            router.push(href);
          }
        }}
      >
        {children}
      </a>
    )
  };
  const copyToClipboard = (str: string): void => {
    void window.navigator.clipboard.writeText(str);
  };

  const handleCopy = (content: string) => {
    copyToClipboard(content);
    setIsCopied(true);
    setTimeout(() => setIsCopied(false), 1000);
  };

  return (
    <>
      {messages.map((m, index) => (
        <ListItem
          key={`${m.id}-${index}`}
          sx={
            m.role === 'user'
              ? messageStyles.userMessage
              : messageStyles.aiMessage
          }
        >
          <Box
            sx={{
              position: 'absolute',
              top: '10px',
              left: '10px'
            }}
          >
            {m.role === 'user' ? (
              <PersonIcon sx={{ color: '#4caf50' }} />
            ) : (
              <AndroidIcon sx={{ color: '#607d8b' }} />
            )}
          </Box>
          {m.role === 'assistant' && (
            <Box
              sx={{
                position: 'absolute',
                top: '5px',
                right: '5px',
                cursor: 'pointer',
                display: 'flex',
                alignItems: 'center',
                justifyContent: 'center',
                width: 24,
                height: 24
              }}
              onClick={() => handleCopy(m.content)}
            >
              {isCopied ? (
                <CheckCircleIcon fontSize="inherit" />
              ) : (
                <ContentCopyIcon fontSize="inherit" />
              )}
            </Box>
          )}
          <Box sx={{ overflowWrap: 'break-word' }}>
            <Typography
              variant="caption"
              sx={{ fontWeight: 'bold', display: 'block' }}
            >
              {m.role === 'user' ? 'You' : 'AI'}
            </Typography>
            {m.role === 'user' ? (
              <ReactMarkdown
                components={componentsUser}
                remarkPlugins={[remarkGfm, remarkMath]}
                rehypePlugins={[rehypeHighlight]}
              >
                {m.content}
              </ReactMarkdown>
            ) : (
              <ReactMarkdown
                components={componentsAI}
                remarkPlugins={[remarkGfm, remarkMath]}
                rehypePlugins={[[rehypeHighlight, highlightOptionsAI]]}
              >
                {m.content}
              </ReactMarkdown>
            )}
          </Box>
        </ListItem>
      ))}
    </>
  );
};

and then in the output return

 <List
          sx={{
            marginBottom: '120px'
          }}
        >
          <ChatMessage messages={messages} />
</List>

This caused exhaustions of the max states depth.

However, after changing the the structure to:

const MemoizedMessage = memo(({ message }: { message: Message }) => {
  const [isCopied, setIsCopied] = useState(false);
  const router = useRouter();
  const componentsAI: Partial<Components> = {
    a: ({ href, children }) => (
      <a
        href={href}
        onClick={(e) => {
          e.preventDefault();
          if (href) {
            router.push(href);
          }
        }}
      >
        {children}
      </a>
    ),
    code({ className, children, ...props }) {
      const match = /language-(\w+)/.exec(className || '');
      const language = match && match[1] ? match[1] : '';
      const inline = !language;
      if (inline) {
        return (
          <code className={className} {...props}>
            {children}
          </code>
        );
      }

      return (
        <div
          style={{
            position: 'relative',
            borderRadius: '5px',
            padding: '20px',
            marginTop: '20px',
            maxWidth: '100%'
          }}
        >
          <span
            style={{
              position: 'absolute',
              top: '0',
              left: '5px',
              fontSize: '0.8em',
              textTransform: 'uppercase'
            }}
          >
            {language}
          </span>
          <div
            style={{
              overflowX: 'auto',
              maxWidth: '1100px'
            }}
          >
            <pre style={{ margin: '0' }}>
              <code className={className} {...props}>
                {children}
              </code>
            </pre>
          </div>
        </div>
      );
    }
  };

  const componentsUser: Partial<Components> = {
    a: ({ href, children }) => (
      <a
        href={href}
        onClick={(e) => {
          e.preventDefault();
          if (href) {
            router.push(href);
          }
        }}
      >
        {children}
      </a>
    )
  };
  const copyToClipboard = (str: string): void => {
    void window.navigator.clipboard.writeText(str);
  };

  const handleCopy = (content: string) => {
    copyToClipboard(content);
    setIsCopied(true);
    setTimeout(() => setIsCopied(false), 1000);
  };

  return (
    <ListItem
      sx={
        message.role === 'user'
          ? messageStyles.userMessage
          : messageStyles.aiMessage
      }
    >
      <Box
        sx={{
          position: 'absolute',
          top: '10px',
          left: '10px'
        }}
      >
        {message.role === 'user' ? (
          <PersonIcon sx={{ color: '#4caf50' }} />
        ) : (
          <AndroidIcon sx={{ color: '#607d8b' }} />
        )}
      </Box>
      {message.role === 'assistant' && (
        <Box
          sx={{
            position: 'absolute',
            top: '5px',
            right: '5px',
            cursor: 'pointer',
            display: 'flex',
            alignItems: 'center',
            justifyContent: 'center',
            width: 24,
            height: 24
          }}
          onClick={() => handleCopy(message.content)}
        >
          {isCopied ? (
            <CheckCircleIcon fontSize="inherit" />
          ) : (
            <ContentCopyIcon fontSize="inherit" />
          )}
        </Box>
      )}
      <Box sx={{ overflowWrap: 'break-word' }}>
        <Typography
          variant="caption"
          sx={{ fontWeight: 'bold', display: 'block' }}
        >
          {message.role === 'user' ? 'You' : 'AI'}
        </Typography>
        {message.role === 'user' ? (
          <ReactMarkdown
            components={componentsUser}
            remarkPlugins={[remarkGfm, remarkMath]}
            rehypePlugins={[rehypeHighlight]}
          >
            {message.content}
          </ReactMarkdown>
        ) : (
          <ReactMarkdown
            components={componentsAI}
            remarkPlugins={[remarkGfm, remarkMath]}
            rehypePlugins={[[rehypeHighlight, highlightOptionsAI]]}
          >
            {message.content}
          </ReactMarkdown>
        )}
      </Box>
    </ListItem>
  );
});
MemoizedMessage.displayName = 'MemoizedMessage';
const ChatMessage: FC<ChatMessageProps> = ({ messages }) => {
  return (
    <>
      {messages.map((message, index) => (
        <MemoizedMessage key={`${message.id}-${index}`} message={message} />
      ))}
    </>
  );
};

the lag issues and the maximum update depth exceeded seems to have disappeared completely. In Dev i was only able to ever get many 4 or 5 messages before chrome just gave up, now i can get pretty much as many as i want!

[lgrammel]

I tried to reproduce the bug with useCompletion and gpt-4o (4096 completion tokens). However, it did not show up for me.

@Jerry-VW Can you provide me with code to reproduce? Ideally some modification of the next/useCompletion example (which I tried w gpt-4o):

• https://github.com/vercel/ai/blob/main/examples/next-openai/app/completion/page.tsx
• https://github.com/vercel/ai/blob/main/examples/next-openai/app/api/completion/route.ts

[ElectricCodeGuy]

I have this older branch of my example project where i have the exact same problem using the "useChat" and an api route.

https://github.com/ElectricCodeGuy/SupabaseAuthWithSSR/tree/0643777a348d63b7d0b0260ad99e5054be1b4062

In dev environment it would always lag out and crash my browser after a few msg. In production i have not experienced the same level of lag. Here it is on par with other chatbots i have tried. So after maybe 20 msg the UI can begin to lock up.
I'm 90% convinced it have something to do with the .map function and re-renders, but i have not successfully found the root cause.

[choipd]

I discovered that when using the streaming method, errors occur when the response message reaches a certain length. Even with streaming, no errors occur if the response message is short. I suspect that this issue arises because the component updates every time a streaming input comes in.

[lgrammel]

I'm looking for a minimal examples because it's unclear to me whether this is an issue with useChat / useCompletion or with the other React code.

@ElectricCodeGuy your examples has a lot of other code, which makes it hard to pinpoint the issue

@Jerry-VW is this for a single response / completion or for a long chat?

@choipd i tried to produce a very long message (max tokens) with no issues. however, i have a pretty fast machine and that might also play a role here

[bioshazard]

I ran into this trying to render chat.messages[0].content as a prop. Fix it with a slice/map:

chat.messages.slice(0,1).map( message => <Component content={message.content} />

I think the problem is React picking up on the .content as a deep dependency to watch for updates. Running it through a map keep the reactivity on the messages maybe.