System Architecture and Storage Model
TheJournalStore crates/palyra-daemon/src/journal.rs#7-12 encapsulates a single SQLite database connection. It operates in Write-Ahead Logging (WAL) mode with foreign key constraints enabled to ensure data integrity and performance under concurrent access crates/palyra-daemon/src/journal.rs#6-9.
Data Flow: Natural Language to Code Entities
The following diagram illustrates how high-level system concepts (Natural Language Space) map to specific implementation entities (Code Entity Space) within the Journal subsystem. Concept to Code Mapping Sources: crates/palyra-daemon/src/journal.rs#1-20, crates/palyra-daemon/src/gateway/runtime.rs#49-53Tamper-Evidence and Hash Chaining
Palyra implements a cryptographic hash chain to make the audit log tamper-evident. Whenhash_chain_enabled is active, every event is linked to its predecessor crates/palyra-daemon/src/journal.rs#11-15.
- Identity Fields: Includes event sequence, type, and actor metadata.
- Payload: The sanitized JSON data of the event.
- Hash Calculation:
hash = SHA-256(prev_hash | identity_fields | payload)crates/palyra-daemon/src/journal.rs#12-14.
JournalHealthReport, which includes a JournalHashChainVerificationReport to detect if any historical records have been modified or deleted crates/palyra-daemon/src/gateway/runtime.rs#49-53.
Event Sanitization and Redaction
To prevent sensitive information (secrets, credentials, PII) from being persisted in the clear, the Journal uses a multi-stage redaction pipeline crates/palyra-daemon/src/journal.rs#11-12.Redaction Logic
- Key-based: Scans JSON keys for fragments like
secret,api_key,token, andpasswordcrates/palyra-daemon/src/journal.rs#81-94. - Value-based: Searches text for phrases such as “bearer token is” or “social security number” crates/palyra-daemon/src/journal.rs#96-130.
- Binary Redaction: Removes raw binary data from payloads to keep the database size manageable.
- Marker: Redacted content is replaced with the string
<redacted>crates/palyra-daemon/src/journal.rs#75-75.
Key Subsystems Managed by JournalStore
The Journal serves as the durable backend for several critical daemon components:| Subsystem | Primary Record Type | Role |
|---|---|---|
| Orchestrator | OrchestratorSessionRecord | Tracks run lifecycle (Pending, InProgress, Done) crates/palyra-daemon/src/orchestrator.rs#22-29. |
| Tape | OrchestratorSessionTranscriptRecord | Append-only log of every interaction in a session crates/palyra-daemon/src/gateway/runtime.rs#76-76. |
| Memory | MemoryItemRecord | Stores facts and vectors for RAG crates/palyra-daemon/src/gateway/runtime.rs#64-65. |
| Approvals | ApprovalDecision | Persists operator decisions for tool execution crates/palyra-daemon/src/lib.rs#148-151. |
| Tool Jobs | ToolJobRecord | Tracks background tool execution and output tails crates/palyra-daemon/src/gateway/runtime.rs#83-85. |
Maintenance and Health
The system includes automated maintenance routines to ensure the database remains healthy and performant.JournalHealthReport
The health report, accessible via theGatewayRuntimeState, monitors:
- WAL Checkpoints: Status of the write-ahead log crates/palyra-daemon/src/gateway/runtime.rs#51-52.
- Hash Integrity: Verification of the SHA-256 chain crates/palyra-daemon/src/gateway/runtime.rs#49-50.
- Migrations: Ensures the schema is at the latest version from the
MIGRATIONSlist crates/palyra-daemon/src/journal.rs#8-9.
Retention and Cleanup
The Journal implements retention policies for artifacts and memory items. For example,MEMORY_RETENTION_DAY_MS defines the base unit for memory aging crates/palyra-daemon/src/journal.rs#150-150. The MemoryPurgeRequest is used to clear expired or explicitly deleted memory segments crates/palyra-daemon/src/lib.rs#150-150.
Sources: crates/palyra-daemon/src/journal.rs#140-160, crates/palyra-daemon/src/gateway/runtime.rs#49-53, crates/palyra-daemon/src/lib.rs#148-153