Skip to main content
The Agent-to-UI (A2UI) protocol is a structured, schema-driven communication layer that allows Palyra agents to dynamically update and interact with the Console UI without requiring custom frontend deployments. It facilitates a “Canvas” experience where the agent can render text, markdown, tables, forms, and charts by sending incremental updates over a gRPC or HTTP stream.

Protocol Architecture

The A2UI protocol is built on three core pillars:
  1. State Synchronization: The UI maintains an A2uiDocument which represents the current view state.
  2. Incremental Patching: Instead of re-sending the entire UI state, the agent sends PatchOperation objects (based on JSON Patch RFC 6902) to modify the existing document.
  3. Security Isolation: The renderer enforces strict sanitization, ensuring that agents cannot execute arbitrary JavaScript or bypass Content Security Policy (CSP) via the UI layer.

Data Flow Diagram

This diagram illustrates how an agent’s intent is transformed into a UI update through the A2UI pipeline. A2UI Update Pipeline Sources: apps/web/src/a2ui/renderer.tsx#36-63, apps/web/src/a2ui/normalize.ts#33-77, apps/web/src/a2ui/tests/renderer.snapshot.test.tsx#26-54

Document Structure and Components

An A2uiDocument consists of a surface identifier, an optional experimental governance block, and an array of components.

Supported Components

The renderer supports a specific set of primitive components defined in the A2uiComponent union type:
Component TypePurposeKey Props
textSimple status or labelstone (normal, success, danger), value
markdownRich text contentvalue (sanitized markdown)
listOrdered or unordered listsitems, ordered (boolean)
tableStructured data gridscolumns, rows
formInteractive data entryfields, submitLabel, title
chartData visualizationseries (label/value pairs), title

Implementation Mapping

This diagram bridges the conceptual UI components to their specific TypeScript implementation entities. Component Implementation Map Sources: apps/web/src/a2ui/renderer.tsx#116-147, apps/web/src/a2ui/normalize.ts#160-194, apps/web/src/a2ui/types.ts#1-24

Patching and Normalization

The A2UI system uses a strict normalization phase before rendering to ensure that incoming JSON from the agent (which may be untrusted or malformed) adheres to safety constraints.

Normalization Logic

The normalizeA2uiDocument function performs the following checks:

Patch Application

Agents send PatchOperation objects to modify the document. Common operations include:
  • add: Appending items to a list or adding a new component.
  • replace: Updating a status message or chart value.
  • remove: Deleting a component after a task is finished.
The applyPatchDocument function applies these operations to the current state before passing the result through the normalization and rendering pipeline apps/web/src/a2ui/tests/renderer.snapshot.test.tsx#47-49.

Security and Sanitization

Security is enforced at the protocol boundary to prevent Cross-Site Scripting (XSS) and unauthorized data exfiltration.

Sanitization Strategies

  1. No Direct HTML: The renderer does not use dangerouslySetInnerHTML. Markdown is rendered via a SanitizedMarkdown component that strips dangerous tags and attributes apps/web/src/a2ui/renderer.tsx#12-12.
  2. Input Coercion: Values are strictly coerced using functions like coerceString, coerceBoolean, and coerceFiniteNumber apps/web/src/a2ui/normalize.ts#4-9.
  3. Identifier Sanitization: Component and field IDs are sanitized to ensure they only contain safe characters for DOM attributes apps/web/src/a2ui/normalize.ts#149-150.
  4. Experimental Consent: Any “ambient” experiment (e.g., push_to_talk) requires explicit consent_required: true in the document, or the normalization will fail apps/web/src/a2ui/normalize.ts#123-128.

Governance Validation

The protocol includes an experimental block that tracks features under development. This block requires: Sources: apps/web/src/a2ui/renderer.tsx#65-96, apps/web/src/a2ui/normalize.ts#79-158, apps/web/src/a2ui/tests/renderer.snapshot.test.tsx#56-80

Interaction and Forms

The A2UI protocol supports bi-directional communication through the form component.
  1. Rendering: A2uiForm renders a set of A2uiFormField entries (text, select, checkbox) apps/web/src/a2ui/renderer.tsx#180-227.
  2. State Management: The form maintains local React state for all field values apps/web/src/a2ui/renderer.tsx#185-196.
  3. Submission: When a user clicks the submit button, an onFormSubmit callback is triggered with the componentId and the collected values apps/web/src/a2ui/renderer.tsx#198-205.
  4. Handoff: These values are typically sent back to the agent via the gRPC RunStream or an HTTP POST to the daemon’s console API.
Sources: apps/web/src/a2ui/renderer.tsx#180-227, apps/web/src/a2ui/types.ts#18-24