Skip to main content
This page describes how the Palyra daemon integrates with AI model providers, manages authentication profiles, prepares complex prompt inputs (including RAG and session history), and orchestrates background routines via the cron scheduler.

​
Model Provider Integration

The daemon supports multiple provider architectures through a unified interface, primarily targeting OpenAI-compatible APIs and Anthropic’s Messages API.

​
Provider Registry & Configuration

The ModelProviderRegistryConfig manages the list of available providers and their associated models. Each provider is defined by a ModelProviderKind (Deterministic, OpenAiCompatible, or Anthropic) crates/palyra-daemon/src/model_provider.rs#37-43. Providers are configured with:

​
Authentication Profiles

Authentication is abstracted through the AuthProfileRegistry. This allows the daemon to refresh credentials (e.g., OAuth2 tokens) independently of the model request logic. When a provider request is initiated, the system resolves the AuthCredential crates/palyra-cli/src/commands/models.rs#189-192.

​
Request Transformation

The daemon transforms internal ProviderRequest structures into provider-specific payloads: Sources: crates/palyra-daemon/src/model_provider.rs#1-205, crates/palyra-cli/src/commands/models.rs#1-124, crates/palyra-daemon/src/config/load.rs#23-28.

​
Provider Input Preparation

Before a message is sent to a model, the prepare_model_provider_input function orchestrates the construction of the final prompt crates/palyra-daemon/src/gateway/tests.rs#50-54.

​
Memory Augmentation (RAG)

If auto-injection is enabled, the daemon performs a vector search via search_memory using the user’s input as a query crates/palyra-daemon/src/application/provider_input.rs#190-192.

​
Session Compaction

To manage long-running conversations within context limits, the daemon employs apply_session_compaction. This process:
  1. Identifies Candidates: Scans the OrchestratorTape for facts, decisions, and summaries crates/palyra-daemon/src/application/session_compaction.rs#158-169.
  2. Summarization: Condenses older turns into a summary_text block crates/palyra-daemon/src/application/session_compaction.rs#82.
  3. Checkpointing: Creates an OrchestratorCheckpointRecord to allow the session to “reset” while retaining critical state crates/palyra-daemon/src/application/session_compaction.rs#21-27.

​
Input Preparation Flow

The following diagram illustrates the transformation from a raw user message to a provider-ready payload. Prompt Assembly Pipeline Sources: crates/palyra-daemon/src/application/provider_input.rs#41-107, crates/palyra-daemon/src/application/session_compaction.rs#31-147, crates/palyra-daemon/src/gateway/tests.rs#50-54.

​
Orchestration & Routines

The cron module handles scheduled execution of agent tasks, known as Routines.

​
Scheduler Loop

The scheduler runs as a background service, polling for due jobs every 15 seconds (SCHEDULER_IDLE_SLEEP) crates/palyra-daemon/src/cron.rs#42.

​
Routine Lifecycle

When a routine is triggered:
  1. Dispatch: A CronRunStartRequest is issued to the journal crates/palyra-daemon/src/cron.rs#35.
  2. Execution: The orchestrator initializes a Run using the routine’s prompt and owner principal crates/palyra-daemon/src/transport/http/handlers/console/routines.rs#56-60.
  3. Finalization: Upon completion, a CronRunFinalizeRequest updates the status to Success or Failure crates/palyra-daemon/src/cron.rs#35.

​
Routine Orchestration Architecture

Sources: crates/palyra-daemon/src/cron.rs#1-156, crates/palyra-daemon/src/transport/http/handlers/console/routines.rs#9-107.

​
Usage Governance & Smart Routing

The usage_governance module monitors token consumption and provides “Smart Routing” to optimize for cost or latency.

​
Budget Evaluation

The system evaluates UsageBudgetPolicyRecord entries against current consumption crates/palyra-daemon/src/usage_governance.rs#11-12.

​
Routing Decisions

The RoutingDecision logic selects the best model based on:
  1. Complexity Score: Estimated based on prompt length and task type crates/palyra-daemon/src/usage_governance.rs#121.
  2. Provider Health: Current latency and error rates crates/palyra-daemon/src/usage_governance.rs#122.
  3. Cost Tier: Low, Standard, or Premium crates/palyra-daemon/src/model_provider.rs#107-111.
Sources: crates/palyra-daemon/src/usage_governance.rs#23-146, crates/palyra-daemon/src/transport/http/handlers/console/usage.rs#13-18.