palyrad daemon responsible for executing time-based workflows, system maintenance tasks, and periodic security audits. It manages the lifecycle of scheduled jobs, enforces concurrency and misfire policies, and integrates with the Cedar policy engine for job authorization.
Scheduler Architecture
The scheduler operates as a background loop spawned during daemon initialization. It manages three primary categories of tasks:- User-Defined Routines: Jobs scheduled via the Cron or Routines API (e.g., “Summarize health every 5 minutes”).
- Memory Maintenance: Periodic tasks for vector database optimization and embedding backfills.
- Security Audits: Background re-verification of installed WASM skills against the
SkillTrustStore.
Scheduler Loop Implementation
The core loop is defined inspawn_scheduler_loop within crates/palyra-daemon/src/cron.rs. It utilizes a tokio::sync::Notify mechanism to wake up either at the next scheduled job time or after a maximum idle sleep of 15 seconds crates/palyra-daemon/src/cron.rs#42-42.
Code Entity Map: Scheduler Flow
The following diagram maps the natural language concepts of “Scheduling” to the specific Rust entities and data structures used in the implementation. Title: Scheduler Entity Mapping Sources: crates/palyra-daemon/src/cron.rs#121-137, crates/palyra-daemon/src/cron.rs#158-174, crates/palyra-daemon/src/journal.rs#35-40Schedule Normalization and Parsing
Palyra supports three types of schedules, which are normalized into a standard internal format usingnormalize_schedule crates/palyra-daemon/src/cron.rs#240-240.
| Schedule Type | Description | Code Representation |
|---|---|---|
| Cron | Standard 5-field cron expression (min, hour, dom, month, dow). | ParsedSchedule::Cron |
| Every | Fixed interval execution (e.g., every 3600s). | ParsedSchedule::Every |
| At | One-off execution at a specific RFC3339 timestamp. | ParsedSchedule::At |
Cron Matching Logic
TheCronMatcher handles complex temporal logic, including support for both UTC and Local timezones crates/palyra-daemon/src/cron.rs#61-83. It implements next_after to calculate the subsequent execution window by iterating through minutes until a match is found, capped by MAX_CRON_LOOKAHEAD_MINUTES crates/palyra-daemon/src/cron.rs#46-46.
Sources: crates/palyra-daemon/src/cron.rs#121-156, crates/palyra-daemon/src/cron.rs#158-174
Job Dispatch and Policy Evaluation
When a job is due, the scheduler callsdispatch_job. This function performs several critical checks before initiating a run.
Cedar Policy Evaluation
Before execution, the scheduler evaluates the job against the security policy usingevaluate_with_context from palyra-policy crates/palyra-daemon/src/cron.rs#14-17. The PolicyRequestContext includes the SYSTEM_DAEMON_PRINCIPAL (“system:daemon”) and the specific routine metadata to ensure the scheduled task has the necessary permissions to invoke the requested tools crates/palyra-daemon/src/cron.rs#55-55.
Execution Policies
The scheduler enforces two primary behavioral policies defined in theCronJobRecord:
- Concurrency Policy (
CronConcurrencyPolicy):Forbid: If a previous run is still active, the new run is skipped.Replace: The active run is cancelled viaOrchestratorCancelRequestbefore starting the new one.QueueOne: (Internal) Queues a single pending execution.
- Misfire Policy (
CronMisfirePolicy):Skip: If the daemon was offline during a scheduled window, ignore the missed tick.CatchUp: Execute the missed tick immediately upon daemon startup.
Background System Tasks
Beyond user routines, the daemon executes internal maintenance tasks on fixed intervals.Memory Maintenance Ticks
TheMemoryStore requires periodic housekeeping to maintain performance and accuracy:
- Maintenance Interval: Every 5 minutes (
MEMORY_MAINTENANCE_INTERVAL) crates/palyra-daemon/src/cron.rs#56-56. This triggers TTL-based eviction and SQLiteVACUUMoperations if configured inFileMemoryRetentionConfigcrates/palyra-common/src/daemon_config_schema.rs#162-167. - Embeddings Backfill: Every 10 minutes (
MEMORY_EMBEDDINGS_BACKFILL_INTERVAL) crates/palyra-daemon/src/cron.rs#57-57. The scheduler identifies text nodes missing vector embeddings and dispatches them to theModelProviderin batches of 64 crates/palyra-daemon/src/cron.rs#58-58.
Skill Re-Audit Jobs
To protect against supply-chain attacks or local tampering, the scheduler performs periodic security re-audits of all installed WASM skills.- Interval: Default is 6 hours (
DEFAULT_SKILL_REAUDIT_INTERVAL), configurable viaPALYRA_SKILL_REAUDIT_INTERVAL_MScrates/palyra-daemon/src/cron.rs#53-54. - Logic: The scheduler iterates through the
InstalledSkillsIndex, re-calculates the SHA-256 hash of the.palyra-skillartifact, and re-runsaudit_skill_artifact_securitycrates/palyra-daemon/src/cron.rs#18-18. If a skill fails the audit, its status is updated toQuarantinein theSkillTrustStore.
Configuration
Scheduler behavior is controlled via thecron and memory.retention sections of palyra.toml.