Skip to main content
The Gateway and Session Orchestration layer is the central processing hub of palyrad. it manages the lifecycle of AI interactions, from receiving inbound messages via gRPC or Channel Connectors to executing multi-step agent “runs” that involve LLM inference, tool execution, and human-in-the-loop approvals.

​
GatewayRuntimeState

The GatewayRuntimeState is the primary container for the daemon’s operational state. It acts as a facade over various subsystems, including the JournalStore, AgentRegistry, ChannelRouter, and ModelProvider.
ComponentRole
JournalStoreHandles persistence of sessions, runs, and the audit-trailed Orchestrator Tape.
AgentRegistryResolves agent definitions and system prompts.
ChannelRouterManages inbound/outbound message routing for platforms like Discord.
ModelProviderInterfaces with LLM backends (OpenAI-compatible or Deterministic).
VaultProvides access to secrets required during tool execution.
Key Code Entities: Sources:
  • crates/palyra-daemon/src/gateway.rs
  • crates/palyra-daemon/src/gateway/runtime.rs

​
Session and Run Lifecycle

Palyra distinguishes between Sessions (long-lived conversational containers) and Runs (individual execution units within a session).

​
Orchestrator Session

A session is identified by a session_id (ULID) and optionally a session_key. It maintains the continuity of a conversation across multiple user interactions. The resolve_orchestrator_session function is used to either retrieve an existing session or initialize a new one based on the incoming message context (Principal, Device, and Channel).

​
Orchestrator Run

A run represents a single “turn” or background task. It is governed by the RunStateMachine, which tracks the lifecycle through various states:
  • Accepted: The run has been created and is awaiting processing.
  • Running: The LLM is generating tokens or a tool is executing.
  • AwaitingApproval: Execution is paused pending human authorization for a sensitive tool call.
  • Succeeded / Failed / Cancelled: Terminal states.
Key Code Entities: Sources:
  • crates/palyra-daemon/src/journal.rs
  • crates/palyra-daemon/src/orchestrator.rs

​
The Orchestrator Tape

The “Tape” is a hash-chained, append-only log of events occurring within a run. Every interaction—user messages, LLM tokens, tool calls, tool outputs, and internal thoughts—is recorded as an OrchestratorTapeRecord.

​
Tape Event Types

  • Message: Inbound user text or outbound agent responses.
  • ToolCallProposal: The LLM requesting to run a specific tool.
  • ToolCallOutput: The result of a tool execution.
  • ModelToken: Raw fragments of text generated by the model.
  • ApprovalRequest: A pause in the tape requiring user intervention.
Key Code Entities: Sources:
  • crates/palyra-daemon/src/journal.rs

​
Message Flow and gRPC Endpoints

The Gateway exposes several gRPC endpoints under the gateway.v1.GatewayService to facilitate message flow.

​
RouteMessage

Used by external connectors (e.g., Discord) to inject messages into the system. The RouteMessage call triggers the ChannelRouter, which determines if the message should start a new run or be ignored based on mention patterns and pairing policies.

​
RunStream

A bidirectional streaming endpoint used by the Web Console and CLI. It streams real-time events (tokens, tool calls) from the RunStateMachine to the client and allows the client to send RunStreamRequest messages (like approvals or cancellations) back to the gateway.

​
AppendEvent

Allows manual insertion of events into a run’s tape, often used for injecting external context or manual overrides.

​
Data Flow Diagram: Inbound Message to LLM

The following diagram bridges the “Natural Language Space” (User Input) to the “Code Entity Space” (Internal Gateway logic). Sources:
  • crates/palyra-daemon/src/transport/grpc/services/gateway/service.rs
  • crates/palyra-daemon/src/gateway.rs
  • crates/palyra-daemon/src/channel_router.rs

​
Orchestration Logic and Tool Execution

When the RunStateMachine encounters a ToolProposal from the ModelProvider, it follows a strict execution flow:
  1. Policy Check: Consults the palyra-policy engine (Cedar) to see if the tool is allowed.
  2. Approval Check: If the tool is marked as sensitive or the policy requires it, the run transitions to AwaitingApproval.
  3. Sandbox Execution: Once approved, the tool is executed via execute_tool_call in the appropriate tier (WASM, Unix rlimit, or Bubblewrap).
  4. Tape Update: The output is appended to the tape and fed back into the LLM for the next turn.
Sources:
  • crates/palyra-daemon/src/orchestrator.rs
  • crates/palyra-daemon/src/gateway/runtime.rs
  • crates/palyra-daemon/src/tool_protocol.rs

​
Key Functions and Classes Reference

SymbolLocationDescription
GatewayRuntimeStatecrates/palyra-daemon/src/gateway.rs#183The main coordinator for daemon operations.
route_messagecrates/palyra-daemon/src/gateway.rs#430Entry point for gRPC RouteMessage requests.
run_streamcrates/palyra-daemon/src/gateway.rs#650Entry point for bidirectional gRPC RunStream.
RunStateMachine::next_transitioncrates/palyra-daemon/src/orchestrator.rs#120Computes the next state transition for a run.
OrchestratorTapeAppendRequestcrates/palyra-daemon/src/journal.rs#720Data structure for adding new events to the audit log.
Sources:
  • crates/palyra-daemon/src/gateway.rs
  • crates/palyra-daemon/src/orchestrator.rs
  • crates/palyra-daemon/src/journal.rs