Memory Scopes and Lifecycle
Palyra enforces a strict hierarchy of memory scopes to ensure data isolation and relevant context injection. Every memory item is owned by aprincipal and can be optionally narrowed to a channel or a specific session crates/palyra-daemon/src/application/memory.rs#11-16.
| Scope | Visibility | Persistence |
|---|---|---|
| Principal | Visible across all channels/sessions for the user. | Permanent (until manual deletion/TTL). |
| Channel | Only visible when the agent is running in the same channel (e.g., specific Discord server). | Permanent for that channel. |
| Session | Only visible within the specific conversation thread (session_id). | Permanent for that session. |
Visibility Enforcement
The system employs a deny-by-default visibility model. Aprincipal mismatch or accessing a channel-scoped item without the matching authenticated channel context results in a PermissionDenied error crates/palyra-daemon/src/application/memory.rs#118-138.
Ingestion Pipeline: MemoryLifecycleProvider
The ingestion of new memory items (viapalyra.memory.retain or automated learning) follows a structured pipeline managed by the MemoryLifecycleProvider crates/palyra-daemon/src/application/memory.rs#4-9.
1. Classification and Risk Assessment
Incoming text is passed toclassify_memory_write to determine:
- Category: Is this a
durable_fact,preference, orprocedure? crates/palyra-daemon/src/application/memory.rs#44. - Sensitivity: Does it contain PII or secrets?
- Approval Gate: High-sensitivity or low-confidence writes are held for operator review crates/palyra-daemon/src/application/memory.rs#5-9.
2. Normalization and Redaction
Text is normalized and passed throughredact_memory_text_for_output, which reuses the journal’s redaction pipeline to strip sensitive URL segments (e.g., tokens, passwords) before the memory is stored or displayed crates/palyra-daemon/src/application/memory.rs#157-170, crates/palyra-daemon/src/sandbox_runner.rs#130-133.
3. Deduplication and Merging
The pipeline merges exact or near-duplicates into existing items to prevent context bloating. It also identifies “correction conflicts” where new information contradicts existing stored facts crates/palyra-daemon/src/application/memory.rs#6-8.Memory Ingestion Data Flow
The following diagram illustrates the path from a tool call to theJournalStore.
Memory Ingestion Flow
Sources: crates/palyra-daemon/src/application/tool_runtime/memory.rs#35-47, crates/palyra-daemon/src/application/memory.rs#1-9.
gRPC MemoryService Interface
TheMemoryService provides the primary interface for the CLI and external integrations to interact with the memory subsystem crates/palyra-daemon/src/application/memory.rs#3-5.
Key RPC methods defined in memory.v1:
IngestMemory: Submits a candidate for the lifecycle pipeline crates/palyra-cli/src/commands/memory.rs#130-133.GetMemoryItem: Retrieves a specific item by itsCanonicalId(ULID) crates/palyra-cli/src/commands/memory.rs#125-128.SearchMemory: Performs hybrid retrieval (lexical + vector) across authorized scopes crates/palyra-daemon/src/application/tool_runtime/memory.rs#106-125.DeleteMemoryItem: Removes an item, enforcingenforce_memory_item_delete_scopecrates/palyra-daemon/src/application/memory.rs#149-155.
Protocol Mapping
The system maps between Protobuf discriminants and internal Rust enums (e.g.,MemorySource) to ensure type safety and allow for “loud failures” if unknown sources are provided crates/palyra-daemon/src/application/memory.rs#72-91.
Storage and Retention Policies
Durable Storage
Memories are persisted in theJournalStore, a SQLite-backed engine. This store handles both the raw content and the FTS5 (Full Text Search) virtual tables used for lexical retrieval crates/palyra-daemon/src/retrieval.rs#3-9.
Retention and TTL
Items can be assigned attl_unix_ms. The palyra.memory.status tool allows agents to inspect:
- Usage vs. hard capacity limits.
- Retention counts per scope.
- Maintenance/Compaction timing crates/palyra-daemon/src/application/tool_registry/builtin.rs#98-103.
Redaction on Egress
Whenever memory is retrieved (for the model or the CLI), it is wrapped in a JSON object and processed byredact_payload_json to ensure no sensitive material leaks into the prompt context or logs crates/palyra-daemon/src/application/memory.rs#161-170.
Retrieval Architecture
Retrieval is “hybrid,” combining lexical scoring, vector embeddings, and recency biases. Retrieval Engine Components Sources: crates/palyra-daemon/src/retrieval.rs#1-10, crates/palyra-daemon/src/application/recall.rs#45-53.Scoring Profiles
Different source kinds (e.g.,Memory vs Transcript) use different scoring weights defined in RetrievalSourceProfileKind crates/palyra-daemon/src/retrieval.rs#154-162. For example, Memory hits often prioritize exact phrase matches and explicit trust labels crates/palyra-daemon/src/application/recall.rs#138-156.
External Indexing
For high-scale environments, the daemon supports anExternalRetrievalRuntime. This allows offloading vector search to external providers while maintaining the JournalStore as the absolute source of truth crates/palyra-daemon/src/retrieval.rs#34-38, crates/palyra-daemon/src/retrieval.rs#103-113.
Sources:
crates/palyra-daemon/src/application/memory.rscrates/palyra-daemon/src/application/tool_runtime/memory.rscrates/palyra-daemon/src/retrieval.rscrates/palyra-daemon/src/application/recall.rscrates/palyra-daemon/src/application/tool_registry/builtin.rscrates/palyra-cli/src/commands/memory.rs