palyrad is responsible for the lifecycle of deferred and recurring operations. It manages three primary categories of work: user-defined Routines (high-level automations), system CronJobs (low-level scheduled tasks), and Background Maintenance (memory vacuuming, embedding backfills, and security re-audits).
Scheduler Architecture
The scheduler operates as a long-running loop spawned during daemon initialization. It utilizes a “poll-and-sleep” model to identify due tasks while maintaining high concurrency for task execution.The Scheduler Loop
The core logic resides inspawn_scheduler_loop, which monitors the JournalStore for jobs whose next_run_at_unix_ms has passed crates/palyra-daemon/src/cron.rs#42-46. It processes due jobs in batches (up to 64) to prevent head-of-line blocking crates/palyra-daemon/src/cron.rs#43.
Scheduler Data Flow
The following diagram illustrates how thespawn_scheduler_loop interacts with the JournalStore and the GatewayRuntimeState to execute tasks.
Scheduler Execution Flow
Sources: crates/palyra-daemon/src/cron.rs#42-100, crates/palyra-daemon/src/journal.rs#34-40
Routines vs. CronJobs
While both represent scheduled work, they exist at different layers of the system.| Feature | CronJob (CronJobRecord) | Routine (RoutineMetadataRecord) |
|---|---|---|
| Definition | Low-level execution primitive. | High-level user automation. |
| Storage | cron_jobs table in SQLite. | definitions.json in routines/ dir. |
| Triggers | Time-based (Cron, Every, At). | Schedule, Hook, Webhook, Manual. |
| Complexity | Simple task metadata. | Includes quiet hours, delivery modes, and templates. |
| Principal | system:cron or system:daemon. | User principal (e.g., user:admin). |
Routine Lifecycle
Routines are managed via theroutines.rs module and synchronized into CronJobRecords when their trigger is time-based crates/palyra-daemon/src/transport/http/handlers/console/routines.rs#160-162.
Routine to Cron Mapping
Sources: crates/palyra-daemon/src/routines.rs#188-203, crates/palyra-daemon/src/cron.rs#85-91, crates/palyra-daemon/src/transport/http/handlers/console/routines.rs#160-170
Execution Policies
The scheduler enforces strict policies to handle missed executions and overlapping runs.Misfire Policies
Defined inCronMisfirePolicy crates/palyra-daemon/src/journal.rs#35:
- Skip: If a job is missed (e.g., daemon was offline), skip the missed window and schedule the next occurrence.
- Catch Up: Execute one run immediately to represent the missed window, then resume normal schedule.
Concurrency Policies
Defined inCronConcurrencyPolicy crates/palyra-daemon/src/journal.rs#35:
- Forbid: If a previous instance of the same job is still
Running, do not start a new one. - Replace: Kill the existing
Runninginstance and start a fresh one. - Queue One: Allow one pending run to wait for the current one to finish.
Background Maintenance Tasks
The daemon runs several internal routines to maintain system health and security. These are hardcoded intervals within thecron.rs module.
Memory Maintenance
- Vacuuming: Every 5 minutes (
MEMORY_MAINTENANCE_INTERVAL), the system checks theFileMemoryRetentionConfigto prune old items based onttl_daysormax_bytescrates/palyra-daemon/src/cron.rs#56, crates/palyra-common/src/daemon_config_schema.rs#162-167. - Embeddings Backfill: Every 10 minutes (
MEMORY_EMBEDDINGS_BACKFILL_INTERVAL), the system identifies memory items missing vector embeddings and batches them for theEmbeddingsProvidercrates/palyra-daemon/src/cron.rs#57-58.
Security Re-Audit
To ensure installed skills remain compliant with evolving policies, the scheduler performs a periodic re-audit of skill artifacts.- Interval: Defaults to 6 hours (
DEFAULT_SKILL_REAUDIT_INTERVAL) crates/palyra-daemon/src/cron.rs#54. - Logic: It iterates through the
InstalledSkillsIndexand callsaudit_skill_artifact_securityon eachartifact.palyra-skillfile crates/palyra-daemon/src/cron.rs#107-119, crates/palyra-daemon/src/cron.rs#18.
Task Summary Table
| Task Name | Interval | Implementation | Purpose |
|---|---|---|---|
| Scheduler Loop | 15s | spawn_scheduler_loop | Dispatches due CronJobs/Routines. |
| Memory Vacuum | 5m | MEMORY_MAINTENANCE_INTERVAL | Enforces retention/TTL policies. |
| Embeddings Backfill | 10m | MEMORY_EMBEDDINGS_BACKFILL_BATCH_SIZE | Generates vectors for RAG. |
| Skill Re-audit | 6h | DEFAULT_SKILL_REAUDIT_INTERVAL | Re-verifies Wasm skill security. |