Skip to main content
The Model Provider system in Palyra serves as the abstraction layer between the orchestrator’s natural language requirements and external LLM APIs. It handles the transformation of internal message structures into provider-specific payloads (OpenAI, Anthropic), manages secure credential storage via the Vault, and implements a robust routing and failover mechanism.

​
Provider Architecture & Input Pipeline

The daemon integrates with LLMs through a unified ModelProvider interface that supports chat completions, embeddings, and audio transcription.

​
Key Data Structures

​
Transformation Pipeline

When a request is sent to a provider, the system performs several transformations:
  1. Vision Handling: If the request contains images, build_openai_chat_content or build_anthropic_messages_payload constructs the appropriate multi-modal payload crates/palyra-daemon/src/model_provider.rs#211-220.
  2. Token Estimation: Before dispatch, estimate_token_count ensures the payload fits within MAX_MODEL_TOKENS_PER_EVENT crates/palyra-daemon/src/model_provider.rs#17-18.
  3. Tool Mapping: Internal tool definitions are converted to provider-specific function/tool schemas.
Natural Language to Code Mapping: Request Dispatch Sources: crates/palyra-daemon/src/model_provider.rs#144-172, crates/palyra-daemon/src/openai_surface.rs#18-40, crates/palyra-auth/src/lib.rs#10-21

​
Auth Profiles & OpenAI OAuth Flow

Palyra uses Auth Profiles to decouple model configuration from sensitive credentials. Credentials are never stored in the main config.toml in plain text; instead, they are stored in the Vault and referenced by a VaultRef crates/palyra-daemon/src/openai_surface.rs#42-48.

​
API Key Connection

The connect_openai_api_key and connect_anthropic_api_key functions validate the key against the provider’s /models endpoint before persisting it to the registry crates/palyra-daemon/src/openai_surface.rs#34-40, crates/palyra-daemon/src/openai_surface.rs#96-102.

​
OAuth Lifecycle

For OpenAI, a PKCE-based OAuth flow is supported to manage short-lived access tokens and long-lived refresh tokens:
  1. Bootstrap: start_openai_oauth_attempt generates a PKCE verifier and challenge crates/palyra-daemon/src/openai_auth.rs#99-107.
  2. Authorize: The user is redirected to the OpenAI authorization URL with a state parameter tracking the attempt_id crates/palyra-daemon/src/openai_auth.rs#109-130.
  3. Callback: The OPENAI_OAUTH_CALLBACK_PATH receives the code and exchanges it for tokens using exchange_authorization_code crates/palyra-daemon/src/openai_auth.rs#132-155.
  4. Persistence: Tokens are stored in the Vault, and an AuthProfileRecord is created in the AuthProfileRegistry crates/palyra-auth/src/lib.rs#10-21.
Auth State Transition
FunctionRoleFile Reference
generate_pkce_verifierCreates entropy for OAuth securitycrates/palyra-daemon/src/openai_auth.rs#99
validate_openai_bearer_tokenProbes /v1/models for key validitycrates/palyra-daemon/src/openai_auth.rs#189
persist_openai_auth_profileSaves metadata to agents.toml or registrycrates/palyra-daemon/src/openai_surface.rs#61
refresh_openai_profileUses refresh token to get new access tokenapps/web/src/console/hooks/useAuthDomain.ts#220
Sources: crates/palyra-daemon/src/openai_auth.rs#11-187, crates/palyra-daemon/src/openai_surface.rs#18-78, apps/web/src/console/hooks/useAuthDomain.ts#168-203

​
Smart Routing & Failover

The ModelProviderRegistryConfig manages how the daemon selects models for different tasks crates/palyra-daemon/src/model_provider.rs#175-187.

​
Cron-Triggered Agent Runs

Model integration extends to background tasks via the Cron system. Agents can be scheduled to run autonomously, utilizing the configured LLM providers.

​
Execution Flow

  1. Scheduler: The CronMatcher identifies jobs due for execution crates/palyra-daemon/src/cron.rs#139-158.
  2. Dispatch: The system triggers a gateway_v1::RunStream request with SYSTEM_DAEMON_PRINCIPAL crates/palyra-daemon/src/cron.rs#55.
  3. Context Injection: The cron job provides the necessary RequestContext, including the agent_id and associated auth_profile_id crates/palyra-daemon/src/cron.rs#29-33.
Code Entity Mapping: Cron to LLM Dispatch Sources: crates/palyra-daemon/src/cron.rs#42-55, crates/palyra-daemon/src/cron.rs#158-174, crates/palyra-daemon/src/model_provider.rs#175-187

​
Configuration & Discovery

Providers and models are configured in the model_provider section of the daemon config crates/palyra-daemon/src/config/load.rs#23-28. Sources: crates/palyra-daemon/src/model_provider.rs#1-205, crates/palyra-daemon/src/config/load.rs#31-45, crates/palyra-daemon/src/openai_surface.rs#1-141