Registry Architecture
The registry is built around theRegistryBackedModelProvider struct, which implements the core logic for routing requests to specific backend implementations based on the ModelProviderKind crates/palyra-daemon/src/model_provider.rs#39-43.
Supported Backend Kinds
| Kind | Description | Protocol |
|---|---|---|
Deterministic | Used for testing or local mocks; returns fixed responses crates/palyra-daemon/src/model_provider.rs#40. | Internal |
OpenAiCompatible | Supports OpenAI’s API and compatible providers (e.g., Together, Groq, LocalAI) crates/palyra-daemon/src/model_provider.rs#41. | REST / JSON |
Anthropic | Dedicated support for Anthropic Messages API crates/palyra-daemon/src/model_provider.rs#42. | REST / JSON |
Data Flow: Model Request Lifecycle
The following diagram illustrates how a request from theOrchestrator is resolved and executed through the registry.
Model Request Resolution Flow
Sources: crates/palyra-daemon/src/model_provider.rs#175-187, crates/palyra-daemon/src/usage_governance.rs#112-130, crates/palyra-daemon/src/model_provider.rs#156-161
Key Components & Implementation
Configuration Schema
The registry is configured via themodel_provider table in palyra.toml. It supports an array of providers and models to allow for complex failover and specialized routing crates/palyra-daemon/src/model_provider.rs#175-187.
- Provider Entry: Defines the connection parameters (
base_url), timeout, and retry logic crates/palyra-daemon/src/model_provider.rs#144-161. - Model Entry: Maps a specific
model_idto aprovider_idand defines its capabilities (vision, tool calls, etc.) crates/palyra-daemon/src/model_provider.rs#164-172.
Smart Routing & Failover
TheUsageGovernance system performs “Smart Routing” by evaluating RoutingDecisionContext crates/palyra-daemon/src/usage_governance.rs#133-146.
- Complexity Scoring: It calculates a score based on prompt tokens and vision inputs crates/palyra-daemon/src/usage_governance.rs#121.
- Health Checks: Providers in a “failed” state are deprioritized crates/palyra-daemon/src/usage_governance.rs#122.
- Failover: If
failover_enabledis true, the registry can switch to a secondary provider if the primary returns a retryable status code (429, 500, 502, 503, 504) crates/palyra-daemon/src/model_provider.rs#24, crates/palyra-daemon/src/model_provider.rs#197.
SSRF Protection (NetGuard)
To prevent Server-Side Request Forgery, the registry uses a validation layer forbase_url. By default, it blocks private network IP ranges unless allow_private_base_url is explicitly enabled for a specific provider crates/palyra-daemon/src/model_provider.rs#149.
- Function:
validate_openai_base_url_network_policycrates/palyra-daemon/src/config/load.rs#24 checks for loopback and private IP addresses.
Token Estimation
The registry includes utilities for estimating token counts without requiring a round-trip to the provider.estimate_token_count: Used for budget enforcement and routing decisions crates/palyra-daemon/src/model_provider.rs#17.MAX_MODEL_TOKENS_PER_EVENT: Enforces a hard limit on event size (default 128k) crates/palyra-daemon/src/model_provider.rs#17.
Authentication & Vault Integration
The registry integrates withpalyra-vault to ensure API keys are never stored in plaintext in the configuration file.
Secret Redaction
Theredact_provider_registry_secrets function crates/palyra-common/src/daemon_config_schema.rs#65-98 ensures that when configuration is exported or logged, sensitive fields like api_key or api_key_vault_ref are replaced with <redacted> crates/palyra-common/src/daemon_config_schema.rs#4.
Auth Profiles
Providers can link to anAuthProfile via auth_profile_id crates/palyra-daemon/src/model_provider.rs#151.
- API Key Flow:
connect_openai_api_keyvalidates the key against the provider’s/v1/modelsendpoint before storing it in the vault crates/palyra-daemon/src/openai_surface.rs#16-46. - OAuth Flow: Supports managed credential rotation for providers like OpenAI crates/palyra-daemon/src/openai_surface.rs#142-186.
CLI Management
Thepalyra models command group provides administrative access to the registry.
| Command | Code Entity | Description |
|---|---|---|
status | load_models_status | Shows current default models and provider health crates/palyra-cli/src/commands/models.rs#196-199. |
list | build_models_list | Lists all models available in the catalog and registry crates/palyra-cli/src/commands/models.rs#201-206. |
set | run_models | Updates the default chat model in palyra.toml crates/palyra-cli/src/commands/models.rs#194-206. |
connect | run_models | Performs a live connectivity check and model discovery crates/palyra-cli/src/commands/models.rs#126-133. |