palyra-daemon. It utilizes a SQLite backend to provide an append-only, verifiable record of all system activity, including agent runs, tool executions, and state transitions. Beyond simple logging, it manages complex relational data for scheduled jobs, human-in-the-loop approvals, and a RAG (Retrieval-Augmented Generation) memory system with vector embedding support.
SQLite Schema and Core Tables
The persistence layer is implemented incrates/palyra-daemon/src/journal.rs using the rusqlite library crates/palyra-daemon/src/journal.rs#13-13. The schema is designed for high-integrity append-only operations and efficient querying of historical state.
Primary Tables
| Table Name | Purpose | Key Columns |
|---|---|---|
journal_events | The primary append-only log of all system events. | id, event_type, payload_json, hash_chain |
memory_items_fts | Full-text search virtual table for long-term memory. | content, tags, source |
canvas_state_snapshots | Periodic snapshots of the dynamic A2UI Canvas state. | canvas_id, snapshot_json, version |
cron_jobs | Definitions and metadata for scheduled background tasks. | job_id, schedule_type, next_run_at |
approvals | Pending and resolved human-in-the-loop requests. | approval_id, subject_type, decision |
Hash-Chain Integrity and Append-Only Triggers
To ensure the audit log cannot be tampered with undetected, thejournal_events table implements a cryptographic hash chain. Every new event record includes a SHA-256 hash that incorporates the hash of the previous record crates/palyra-daemon/src/journal.rs#16-16.
Data Flow: Event Persistence
TheJournalStore::append function handles the creation of new events. It ensures that sensitive information (like API keys or passwords) is redacted before being written to disk using a predefined list of SENSITIVE_KEY_FRAGMENTS crates/palyra-daemon/src/journal.rs#31-44.
System Component Interaction
Sources: crates/palyra-daemon/src/gateway.rs#63-71, crates/palyra-daemon/src/journal.rs#29-45
Memory Embedding and Recall System
Palyra implements a RAG-capable memory system. It supports both Full-Text Search (FTS) and vector-based similarity search for retrieving relevant context during agent runs.Embedding Providers
The system uses theMemoryEmbeddingProvider trait to abstract the embedding logic crates/palyra-daemon/src/journal.rs#64-68.
- HashMemoryEmbeddingProvider: A deterministic provider that creates vector representations based on text hashing, used for local-first or air-gapped environments crates/palyra-daemon/src/journal.rs#71-100.
- External Providers: The
gatewaycan bridge to high-dimensional models (e.g., OpenAI) via theModelProviderinterface crates/palyra-daemon/src/gateway.rs#73-76.
Recall Logic
When an agent requests memory recall, theJournalStore performs a two-stage search:
- FTS Match: Queries the
memory_items_ftstable for keyword relevance. - Vector Search: Calculates the cosine similarity between the query embedding and stored item embeddings crates/palyra-daemon/src/journal.rs#51-53.
Canvas State and Snapshots
The “Canvas” (A2UI) represents a dynamic UI surface that agents can manipulate. To allow for session recovery and “time-travel” debugging, theJournalStore persists both incremental patches and full state snapshots.
- Patches: Small JSON-patch objects stored as
CanvasStatePatchRecordcrates/palyra-daemon/src/journal.rs#59-60. - Snapshots: Full state representations captured every patches or upon specific lifecycle events to prevent long replay chains crates/palyra-daemon/src/gateway.rs#139-140.
Scheduled Tasks (Cron) Persistence
TheJournalStore manages the lifecycle of CronJobRecord entities. It tracks the next_run_at_unix_ms and enforces CronConcurrencyPolicy (e.g., Forbid, Replace, QueueOne) crates/palyra-daemon/src/journal.rs#130-156. The scheduler loop (described in section 2.5) relies on the JournalStore to atomically claim jobs for execution to prevent double-firing in multi-node setups.
| Function | File | Description |
|---|---|---|
create_cron_job | journal.rs | Persists a new job definition with its schedule. |
start_cron_run | journal.rs | Transitions a job to Running and records the start time. |
finalize_cron_run | journal.rs | Records the Succeeded or Failed outcome and calculates the next run time. |