GatewayRuntimeState
TheGatewayRuntimeState is the central shared state object for the gateway. It acts as a facade over the various subsystems required for orchestration, including the JournalStore, AgentRegistry, ChannelRouter, and ModelProvider crates/palyra-daemon/src/gateway.rs#75-82.
Key responsibilities include:
- Run Management: Tracking active
RunStateMachineinstances and their transitions crates/palyra-daemon/src/gateway.rs#77. - Resource Routing: Dispatching messages through the
ChannelRoutercrates/palyra-daemon/src/gateway.rs#50-54. - Policy Enforcement: Evaluating requests against the Cedar-based policy engine crates/palyra-daemon/src/gateway.rs#23.
- Caching: Maintaining TTL-based caches for tool approvals and memory search results to minimize latency crates/palyra-daemon/src/gateway/runtime.rs#145-164.
Configuration Snapshots
The engine operates usingGatewayRuntimeConfigSnapshot, which defines operational limits such as MAX_TAPE_ENTRIES_PER_RESPONSE and budget constraints for tool execution crates/palyra-daemon/src/gateway/runtime.rs#36-55.
Sources: crates/palyra-daemon/src/gateway.rs#1-110, crates/palyra-daemon/src/gateway/runtime.rs#35-164
The Run State Machine
Every interaction in Palyra is encapsulated in a Run. TheRunStateMachine governs the lifecycle of these runs, ensuring they move through valid states.
| State | Description |
|---|---|
Pending | Initial state before execution starts. |
InProgress | The run is actively processing (LLM is generating or tool is executing). |
Done | Successful completion of the run. |
Cancelled | Execution stopped due to user request or timeout. |
Failed | Terminal state following an unrecoverable error. |
RunTransition events (e.g., StartStreaming, Complete, Cancel) crates/palyra-daemon/src/application/run_stream/orchestration.rs#197-202.
Sources: crates/palyra-daemon/src/orchestrator.rs#26-28, crates/palyra-daemon/src/application/run_stream/orchestration.rs#106-143
Agentic Execution Loop
The orchestration engine follows a “Route → LLM → Tool → Approval” loop. This process is primarily managed inorchestration.rs and provider_events.rs.
Execution Flow Diagram
1. Message Routing
When a message arrives viaRouteMessage, the ChannelRouter determines which agent should handle the input based on pairings and session context crates/palyra-daemon/src/gateway.rs#50-54.
2. LLM Interaction
The engine prepares the prompt viaprepare_model_provider_input, which injects relevant memories from the JournalStore and current session state crates/palyra-daemon/src/application/run_stream/orchestration.rs#15-17.
3. Tool Dispatch & Approval Loop
If the LLM requests a tool call:- Policy Check: The engine checks
ToolCallPolicySnapshot. If the tool is marked as sensitive, it triggers anApprovalPromptRecordcrates/palyra-daemon/src/gateway.rs#78-81. - Approval: The run enters a waiting state until a client provides an
ApprovalDecisionvia theApprovalsServicecrates/palyra-daemon/src/gateway.rs#57-59. - Execution: Once approved,
execute_tool_callruns the tool (often in a Wasm or Tier-C sandbox) crates/palyra-daemon/src/gateway.rs#78-79.
RunStream gRPC Endpoint
TheRunStream endpoint in gateway.proto is the primary bi-directional interface for real-time agent interaction.
Data Flow and Persistence
Key Functions
finalize_run_stream_after_provider_response: Transitions the state machine toDoneand clears self-healing heartbeats crates/palyra-daemon/src/application/run_stream/orchestration.rs#106-143.execute_run_stream_provider_request: Wraps the LLM call in a cancellation-aware loop that pollsis_orchestrator_cancel_requestedevery 100ms crates/palyra-daemon/src/application/run_stream/orchestration.rs#146-182.ensure_run_stream_in_progress: Ensures theInProgressstate is persisted and emitted to the client exactly once per run crates/palyra-daemon/src/application/run_stream/orchestration.rs#185-202.
Tape and Event Sequencing
Every event emitted in aRunStream is recorded in the “Tape”—an append-only log of a specific run’s events. The engine tracks tape_seq to ensure clients receive events in the correct order even if they reconnect crates/palyra-daemon/src/application/run_stream/orchestration.rs#111-135.
Sources: crates/palyra-daemon/src/application/run_stream/orchestration.rs#106-202, crates/palyra-daemon/src/gateway.rs#84-110, crates/palyra-daemon/src/journal.rs#63-70