A2UI replies

In a session created with "output_format": "a2ui" (the default), a reply can come as UI as well as words: cards, lists, images, buttons and forms that your app renders natively. The UI is described in A2UI — an open protocol for UI that an assistant writes and a client renders — version 1.0, with its basic catalog of components (Text, Image, Icon, Video, AudioPlayer, Row, Column, List, Card, Tabs, Modal, Divider, Button, TextField, CheckBox, ChoicePicker, Slider, DateTimeInput).

Any A2UI 1.0 renderer that supports the basic catalog can render the replies.

Where the UI arrives

• ui.updated frames on the event stream carry a task's whole UI data each time it changes: a list of A2UI messages. Replace what you rendered for that task with the new list — never merge it.
• The end frame (task.completed, or task.stopped) carries the final ui beside the reply's text.
• The text of an a2ui reply is plain text: the words the reply says outside its UI. Show it as text next to the UI.

{
  "task_id": "exec-5a2c…",
  "ui": [
    {
      "version": "v1.0",
      "createSurface": {
        "surfaceId": "exec-5a2c…/t0/main",
        "catalogId": "https://a2ui.org/specification/v1_0/catalogs/basic/catalog.json"
      }
    },
    {
      "version": "v1.0",
      "updateComponents": {
        "surfaceId": "exec-5a2c…/t0/main",
        "components": [
          { "id": "root", "component": "Card", "child": "body" },
          { "id": "body", "component": "Column", "children": ["title", "photo", "more"] },
          { "id": "title", "component": "Text", "text": "**Leek and potato soup**" },
          { "id": "photo", "component": "Image", "url": { "path": "/media/s0/url" } },
          {
            "id": "more",
            "component": "Button",
            "child": "more_label",
            "action": { "event": { "name": "show_steps", "context": {} } }
          },
          { "id": "more_label", "component": "Text", "text": "Show the steps" }
        ]
      }
    },
    {
      "version": "v1.0",
      "updateDataModel": {
        "surfaceId": "exec-5a2c…/t0/main",
        "path": "/media/s0",
        "value": { "url": "https://…", "status": "ready", "kind": "image" }
      }
    }
  ]
}

• A surface's surfaceId is unique within the session. Treat it as an opaque string.
• Media the task makes appear in the surface's data model under /media/<slot>, with a url (a signed download link that expires), a status and a kind. Components bind to them, so an image shows once its generation finishes — that change arrives as a new ui.updated.
• The same media are also reported, one by one, in media.updated frames.

Send the user's action back

When the user acts on a component that has an action — taps a Button, submits a form — your renderer produces an A2UI action message. Send it, as it is, in a2ui_action with the message the user sees themselves send (such as the button's label) as text:

curl -s "$NEWEYES_API_BASE/v1/sessions/$SESSION_ID/messages" -X POST \
  -H "Authorization: Bearer $NEWEYES_USER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Show the steps",
    "a2ui_action": {
      "version": "v1.0",
      "action": {
        "name": "show_steps",
        "surfaceId": "exec-5a2c…/t0/main",
        "sourceComponentId": "more",
        "timestamp": "2026-10-19T08:03:00Z",
        "context": {}
      }
    }
  }'

• action.name is the component's action.event.name; surfaceId and sourceComponentId say where the user acted.
• action.context holds the event's context with its data bindings resolved: the values the user picked or entered. It may be up to 4,000 characters as JSON, nested up to 8 levels.
• Other fields the A2UI specification allows in the action message are accepted.
• Only an a2ui session takes an a2ui_action; a markdown session refuses it with 422 validation_failed.

The message starts a task like any other, and its reply streams the same way.

Choosing the output format

	a2ui	markdown
Reply	UI data plus plain text	Markdown text
Your app needs	An A2UI renderer	A Markdown renderer
Interactive elements	Buttons, forms, pickers — sent back as a2ui_action	None: the user types the next message

The format is fixed when the session is created.