JournalStore is the central persistence authority for the Palyra daemon. It implements an event-sourcing architecture backed by SQLite, responsible for recording every interaction, tool execution, and state change within the system. Beyond simple logging, it manages complex relational data for agent sessions, cron schedules, and a vector-enabled memory subsystem for RAG (Retrieval-Augmented Generation).
1. Architecture and Event Sourcing
The system utilizes an event-sourcing pattern where the state of a “Run” or “Session” is derived from a sequence of immutable events. Each event is assigned a unique ULID and is associated with a specificsession_id and run_id.
Hash-Chaining and Integrity
To ensure the auditability of the journal, Palyra supports hash-chaining for specific export types, such as approvals. This creates a cryptographic link between records, starting from a known seed.- Seed Hex:
0000000000000000000000000000000000000000000000000000000000000000crates/palyra-daemon/src/gateway.rs#114-115. - Hashing: The system uses
Sha256to compute digests of event payloads crates/palyra-daemon/src/gateway.rs#31-31.
Key Entities Mapping
The following diagram bridges the high-level persistence concepts to the underlying code entities. Diagram: Persistence Layer Entity Mapping Sources: crates/palyra-daemon/src/journal.rs#68-71, crates/palyra-daemon/src/gateway.rs#56-712. Data Flow: Recording an Event
When the gateway receives a request to append an event via gRPCAppendEvent, it validates the protocol version and the canonical IDs before committing to the JournalStore.
Execution Flow
- gRPC Entry:
GatewayServiceImpl::append_eventreceivesAppendEventRequestcrates/palyra-daemon/src/transport/grpc/services/gateway/service.rs#117-120. - Validation: Checks
CANONICAL_PROTOCOL_MAJORand validates ULIDs crates/palyra-daemon/src/transport/grpc/services/gateway/service.rs#125-142. - Journal Dispatch: Calls
state.record_journal_eventwhich wrapsJournalStore::appendcrates/palyra-daemon/src/transport/grpc/services/gateway/service.rs#155-168. - Persistence: The
JournalStoreexecutes a SQLINSERTinto thejournal_eventstable within a transaction crates/palyra-daemon/src/journal.rs#13-13.
Constraints and Budgets
The system enforces strict latency and size constraints to ensure performance:| Constraint | Value | Source |
|---|---|---|
| Journal Write Latency Budget | 25ms | gateway.rs#92 |
| Max Recent Events Cache | 100 | gateway.rs#89 |
| Max Tape Entries per Run | 1,024 | gateway.rs#103 |
3. Canvas State and Compaction
The “Canvas” represents the persistent UI state shared between the agent and the user. Instead of storing the full state every time, Palyra uses a combination of snapshots and JSON patches.CanvasStateSnapshotRecord: Stores the full base state at a specific point in time crates/palyra-daemon/src/gateway.rs#59-59.CanvasStatePatchRecord: Stores incremental changes usingRFC 6902JSON Patches crates/palyra-daemon/src/gateway.rs#59-59.- Compaction: Periodically, the system collapses patches into a new snapshot to optimize loading times. The system limits the number of recovery snapshots to 10,000 crates/palyra-daemon/src/gateway.rs#139-139.
apply_patch_document and build_replace_root_patch from the palyra-a2ui crate are used to manage these transitions crates/palyra-daemon/src/gateway.rs#13-15.
Sources: crates/palyra-daemon/src/gateway.rs#13-60, crates/palyra-daemon/src/journal.rs#12-12
4. Memory Subsystem (RAG)
The Memory subsystem provides the daemon with long-term retrieval capabilities. It supports vector search (semantic) and tag-based filtering.Embedding Pipeline
TheJournalStore integrates with a MemoryEmbeddingProvider. By default, it uses a HashMemoryEmbeddingProvider for deterministic testing, but it is designed to interface with model providers like OpenAI or Anthropic for production use.
- Default Dimensions: 64 crates/palyra-daemon/src/journal.rs#53-53.
- Maintenance: A background task runs every 5 minutes (
MEMORY_MAINTENANCE_INTERVAL) to handle TTL expiration and vacuuming crates/palyra-daemon/src/cron.rs#56-56. - Backfill: A separate loop runs every 10 minutes to generate embeddings for items created without them crates/palyra-daemon/src/cron.rs#57-57.
5. Session and Run Lifecycle Persistence
TheJournalStore tracks the lifecycle of every agent interaction through the OrchestratorSessionRecord and CronRunRecord.
Session States
Sessions transition through various states, often influenced by theRunStateMachine crates/palyra-daemon/src/gateway.rs#77-77.
OrchestratorSessionRecord: Contains metadata likesession_key,title, andbranch_statecrates/palyra-daemon/src/gateway/messages.rs#6-38.CronJobRecord: Manages scheduled tasks, including theirconcurrency_policy(Forbid, Replace, QueueOne) andmisfire_policycrates/palyra-daemon/src/journal.rs#130-164.
Data Retention and Cleanup
Retention is configured viaFileMemoryRetentionConfig crates/palyra-common/src/daemon_config_schema.rs#199-205. The daemon supports:
- TTL-based Purge: Automatically deleting items older than a specified number of days crates/palyra-daemon/src/journal.rs#56-56.
- Capacity-based Purge: Limiting the total number of entries or bytes in the memory store crates/palyra-common/src/daemon_config_schema.rs#201-202.