RunStream lifecycle, transforming raw gRPC requests into managed agentic operations.
Orchestration & RunStream Lifecycle
The orchestration logic incrates/palyra-daemon/src/application/run_stream/orchestration.rs manages the state machine of a “Run”. It coordinates model provider requests, handles streaming events, and ensures that the system remains responsive to cancellation signals.
Execution Flow
- Input Preparation: The orchestrator calls
prepare_model_provider_inputto aggregate session history, context references, and system prompts crates/palyra-daemon/src/application/run_stream/orchestration.rs#16-18. - Usage Routing: It calculates the
UsageRoutingPlanto determine which model provider should handle the request based on governance policies crates/palyra-daemon/src/application/run_stream/orchestration.rs#32. - Provider Execution: The
execute_run_stream_provider_requestfunction wraps the model provider call in a loop that polls for orchestrator cancellation every 100ms crates/palyra-daemon/src/application/run_stream/orchestration.rs#147-159. - Event Processing: Provider outputs are processed via
process_run_stream_provider_events, which handles tool calls, content generation, and metadata updates crates/palyra-daemon/src/application/run_stream/orchestration.rs#13-15. - Finalization: After the provider completes, the run is transitioned to a
Donestate, and post-run reflection is scheduled crates/palyra-daemon/src/application/run_stream/orchestration.rs#107-144.
Run Lifecycle State Transitions
TheRunStateMachine tracks the progress of a run through several key states defined in crates/palyra-daemon/src/orchestrator.rs.
| State | Transition Trigger | Description |
|---|---|---|
Pending | Initial | The run is created but not yet active. |
InProgress | StartStreaming | The orchestrator has begun communicating with a model provider. |
Done | Complete | The run finished successfully. |
Cancelled | Cancel | The user or system terminated the run mid-execution. |
Tool Flow & Approvals
Tool execution within aRunStream is governed by a security and approval pipeline located in crates/palyra-daemon/src/application/run_stream/tool_flow.rs.
Tool Execution Pipeline
The functionprocess_run_stream_tool_proposal_event manages the lifecycle of a single tool call crates/palyra-daemon/src/application/run_stream/tool_flow.rs#53-69:
- Security Evaluation:
evaluate_tool_proposal_securitychecks the tool against Cedar policies and skill-specific constraints crates/palyra-daemon/src/application/run_stream/tool_flow.rs#121-134. - Approval Resolution: If a tool is deemed sensitive or requires explicit permission, the system checks for a cached decision or pauses for a human-in-the-loop (HITL) response via
resolve_run_stream_tool_approval_outcomecrates/palyra-daemon/src/application/run_stream/tool_flow.rs#212-228. - Execution: Once approved,
execute_run_stream_tool_proposaldispatches the call to the appropriate runtime (WASM, Shell, or Internal) crates/palyra-daemon/src/application/run_stream/tool_flow.rs#87-100. - Tape Recording: Every step (Proposal, Decision, Result) is sent to the client via the
Tapeto ensure the UI can render the tool’s progress in real-time crates/palyra-daemon/src/application/run_stream/tool_flow.rs#38-43.
Data Flow: Tool Proposal to Result
Sources: crates/palyra-daemon/src/application/run_stream/tool_flow.rs#105-208, crates/palyra-daemon/src/application/run_stream/tool_flow.rs#147-162.Context References
Palyra allows users to inject external context into a prompt using special syntax (e.g.,@file, @url). The application/context_references.rs module handles the resolution and rendering of these references.
- Resolution:
preview_context_referencesparses the input text and fetches the content for up to 8 references crates/palyra-daemon/src/application/context_references.rs#72-129. - Resource Fetching: It supports various types including local files, folders, git diffs, and URLs (using
execute_http_fetch_tool) crates/palyra-daemon/src/application/context_references.rs#187-206. - Prompt Injection:
render_context_reference_promptwraps the resolved content in<context_references>XML tags for the LLM to consume crates/palyra-daemon/src/application/context_references.rs#131-176.
Learning & Reflection Pipeline
The learning pipeline is an autonomous background process that extracts “Durable Facts”, “Preferences”, and “Procedures” from chat history.Post-Run Reflection
When a run completes,schedule_post_run_reflection creates a background task crates/palyra-daemon/src/application/learning.rs#41-46. This task is picked up by the BackgroundQueue and processed by process_post_run_reflection_task crates/palyra-daemon/src/application/learning.rs#125-128.
Learning Candidates
The reflection process generates three types of candidates:- Compaction Candidates: Summaries of long conversations to save context tokens crates/palyra-daemon/src/application/learning.rs#161-168.
- Preference Candidates: User-specific stylistic or operational choices crates/palyra-daemon/src/application/learning.rs#169-176.
- Procedure Candidates: Reusable sequences of tool calls that the agent has successfully performed crates/palyra-daemon/src/application/learning.rs#177-185.
Confidence Thresholds
Learning is governed by BPS (basis points) thresholds defined inLearningRuntimeConfig crates/palyra-daemon/src/gateway/runtime.rs#85-96. If a candidate’s confidence exceeds the auto_write_threshold_bps, it is applied immediately; otherwise, it remains in PendingReview for the user crates/palyra-daemon/src/application/learning.rs#189-195.
Sources: crates/palyra-daemon/src/application/learning.rs#30-32, crates/palyra-daemon/src/gateway/runtime.rs#151-166.
Background Queue & Self-Healing
TheBackgroundQueue serves as the executor for non-interactive tasks like learning, session cleanup, and scheduled maintenance.
Task Processing
spawn_background_queue_loop initializes a long-running task that polls the JournalStore for pending OrchestratorBackgroundTaskRecord entries crates/palyra-daemon/src/background_queue.rs#31-44. Tasks are processed with heartbeats to allow the self-healing system to detect “stuck” work crates/palyra-daemon/src/background_queue.rs#108-112.
Self-Healing Watchdog
TheSelfHealingState tracks incidents across domains like Browser, Artifact, and Approval crates/palyra-daemon/src/self_healing.rs#37-42.
- Stuck Detection: If a run or background task hasn’t updated its heartbeat within a specific window (e.g., 120 seconds for runs), the watchdog marks it as a
stuckincident crates/palyra-daemon/src/self_healing.rs#25-27. - Remediation: Depending on the
SelfHealingMode(Disabled, ObserveOnly, or Auto), the system may automatically attempt to cancel stuck runs or restart degraded services crates/palyra-daemon/src/self_healing.rs#79-83.