> ## Documentation Index
> Fetch the complete documentation index at: https://docs-code.palyra.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Memory Lifecycle and Storage

<details>
  <summary>Relevant source files</summary>

  The following files were used as context for generating this wiki page:

  * crates/palyra-cli/src/args/memory.rs
  * crates/palyra-cli/src/args/tests/parser\_stability\_tests.rs
  * crates/palyra-cli/src/commands/memory.rs
  * crates/palyra-cli/src/commands/memory\_external\_index.rs
  * crates/palyra-cli/tests/help\_snapshots/memory-help.txt
  * crates/palyra-cli/tests/help\_snapshots/memory-learning-help.txt
  * crates/palyra-common/src/process\_runner\_input.rs
  * crates/palyra-daemon/src/application/instruction\_compiler.rs
  * crates/palyra-daemon/src/application/memory.rs
  * crates/palyra-daemon/src/application/memory\_provider.rs
  * crates/palyra-daemon/src/application/recall.rs
  * crates/palyra-daemon/src/application/service\_authorization.rs
  * crates/palyra-daemon/src/application/tool\_registry/builtin.rs
  * crates/palyra-daemon/src/application/tool\_registry/tests.rs
  * crates/palyra-daemon/src/application/tool\_runtime/memory.rs
  * crates/palyra-daemon/src/gateway/runtime/external\_retrieval.rs
  * crates/palyra-daemon/src/gateway/tests.rs
  * crates/palyra-daemon/src/journal/retrieval\_index\_status.rs
  * crates/palyra-daemon/src/provider\_leases.rs
  * crates/palyra-daemon/src/retrieval.rs
  * crates/palyra-daemon/src/retrieval/external\_index.rs
  * crates/palyra-daemon/src/sandbox\_runner.rs
  * crates/palyra-daemon/src/transport/grpc/services/memory/service.rs
  * crates/palyra-daemon/src/transport/http/handlers/console/memory\_external\_index.rs
  * crates/palyra-daemon/src/transport/http/handlers/console/usage.rs
  * crates/palyra-daemon/src/usage\_governance.rs
  * schemas/proto/palyra/v1/memory.proto
</details>

The memory subsystem in Palyra manages the ingestion, classification, storage, and retrieval of agent-relevant facts and preferences. It bridges transient conversation state with durable principal-owned knowledge using a multi-scoped architecture.

## Memory Scopes and Lifecycle

Palyra enforces a strict hierarchy of memory scopes to ensure data isolation and relevant context injection. Every memory item is owned by a `principal` and can be optionally narrowed to a `channel` or a specific `session` [crates/palyra-daemon/src/application/memory.rs#11-16](http://crates/palyra-daemon/src/application/memory.rs#11-16).

| Scope         | Visibility                                                                                  | Persistence                            |
| :------------ | :------------------------------------------------------------------------------------------ | :------------------------------------- |
| **Principal** | Visible across all channels/sessions for the user.                                          | Permanent (until manual deletion/TTL). |
| **Channel**   | Only visible when the agent is running in the same channel (e.g., specific Discord server). | Permanent for that channel.            |
| **Session**   | Only visible within the specific conversation thread (`session_id`).                        | Permanent for that session.            |

### Visibility Enforcement

The system employs a deny-by-default visibility model. A `principal` mismatch or accessing a `channel-scoped` item without the matching authenticated channel context results in a `PermissionDenied` error [crates/palyra-daemon/src/application/memory.rs#118-138](http://crates/palyra-daemon/src/application/memory.rs#118-138).

## Ingestion Pipeline: MemoryLifecycleProvider

The ingestion of new memory items (via `palyra.memory.retain` or automated learning) follows a structured pipeline managed by the `MemoryLifecycleProvider` [crates/palyra-daemon/src/application/memory.rs#4-9](http://crates/palyra-daemon/src/application/memory.rs#4-9).

### 1. Classification and Risk Assessment

Incoming text is passed to `classify_memory_write` to determine:

* **Category**: Is this a `durable_fact`, `preference`, or `procedure`? [crates/palyra-daemon/src/application/memory.rs#44](http://crates/palyra-daemon/src/application/memory.rs#44).
* **Sensitivity**: Does it contain PII or secrets?
* **Approval Gate**: High-sensitivity or low-confidence writes are held for operator review [crates/palyra-daemon/src/application/memory.rs#5-9](http://crates/palyra-daemon/src/application/memory.rs#5-9).

### 2. Normalization and Redaction

Text is normalized and passed through `redact_memory_text_for_output`, which reuses the journal's redaction pipeline to strip sensitive URL segments (e.g., tokens, passwords) before the memory is stored or displayed [crates/palyra-daemon/src/application/memory.rs#157-170](http://crates/palyra-daemon/src/application/memory.rs#157-170), [crates/palyra-daemon/src/sandbox\_runner.rs#130-133](http://crates/palyra-daemon/src/sandbox_runner.rs#130-133).

### 3. Deduplication and Merging

The pipeline merges exact or near-duplicates into existing items to prevent context bloating. It also identifies "correction conflicts" where new information contradicts existing stored facts [crates/palyra-daemon/src/application/memory.rs#6-8](http://crates/palyra-daemon/src/application/memory.rs#6-8).

### Memory Ingestion Data Flow

The following diagram illustrates the path from a tool call to the `JournalStore`.

**Memory Ingestion Flow**

```mermaid theme={null}
graph TD
    subgraph "Tool Runtime Space"
        A["palyra.memory.retain"] --> B["execute_memory_retain_tool"]
    end

    subgraph "Memory Application Space"
        B --> C["MemoryLifecycleProvider::retain_memory_candidate"]
        C --> D["classify_memory_write"]
        D --> E["redact_memory_text_for_output"]
        E --> F["Deduplication / Merge Logic"]
    end

    subgraph "Storage Space"
        F --> G["GatewayRuntimeState::ingest_memory"]
        G --> H[("JournalStore (SQLite)")]
    end

    style H stroke-width:2px
```

Sources: [crates/palyra-daemon/src/application/tool\_runtime/memory.rs#35-47](http://crates/palyra-daemon/src/application/tool_runtime/memory.rs#35-47), [crates/palyra-daemon/src/application/memory.rs#1-9](http://crates/palyra-daemon/src/application/memory.rs#1-9).

## gRPC MemoryService Interface

The `MemoryService` provides the primary interface for the CLI and external integrations to interact with the memory subsystem [crates/palyra-daemon/src/application/memory.rs#3-5](http://crates/palyra-daemon/src/application/memory.rs#3-5).

Key RPC methods defined in `memory.v1`:

* `IngestMemory`: Submits a candidate for the lifecycle pipeline [crates/palyra-cli/src/commands/memory.rs#130-133](http://crates/palyra-cli/src/commands/memory.rs#130-133).
* `GetMemoryItem`: Retrieves a specific item by its `CanonicalId` (ULID) [crates/palyra-cli/src/commands/memory.rs#125-128](http://crates/palyra-cli/src/commands/memory.rs#125-128).
* `SearchMemory`: Performs hybrid retrieval (lexical + vector) across authorized scopes [crates/palyra-daemon/src/application/tool\_runtime/memory.rs#106-125](http://crates/palyra-daemon/src/application/tool_runtime/memory.rs#106-125).
* `DeleteMemoryItem`: Removes an item, enforcing `enforce_memory_item_delete_scope` [crates/palyra-daemon/src/application/memory.rs#149-155](http://crates/palyra-daemon/src/application/memory.rs#149-155).

### Protocol Mapping

The system maps between Protobuf discriminants and internal Rust enums (e.g., `MemorySource`) to ensure type safety and allow for "loud failures" if unknown sources are provided [crates/palyra-daemon/src/application/memory.rs#72-91](http://crates/palyra-daemon/src/application/memory.rs#72-91).

## Storage and Retention Policies

### Durable Storage

Memories are persisted in the `JournalStore`, a SQLite-backed engine. This store handles both the raw content and the FTS5 (Full Text Search) virtual tables used for lexical retrieval [crates/palyra-daemon/src/retrieval.rs#3-9](http://crates/palyra-daemon/src/retrieval.rs#3-9).

### Retention and TTL

Items can be assigned a `ttl_unix_ms`. The `palyra.memory.status` tool allows agents to inspect:

* Usage vs. hard capacity limits.
* Retention counts per scope.
* Maintenance/Compaction timing [crates/palyra-daemon/src/application/tool\_registry/builtin.rs#98-103](http://crates/palyra-daemon/src/application/tool_registry/builtin.rs#98-103).

### Redaction on Egress

Whenever memory is retrieved (for the model or the CLI), it is wrapped in a JSON object and processed by `redact_payload_json` to ensure no sensitive material leaks into the prompt context or logs [crates/palyra-daemon/src/application/memory.rs#161-170](http://crates/palyra-daemon/src/application/memory.rs#161-170).

## Retrieval Architecture

Retrieval is "hybrid," combining lexical scoring, vector embeddings, and recency biases.

**Retrieval Engine Components**

```mermaid theme={null}
graph LR
    subgraph "Request Layer"
        R["MemorySearchRequest"]
    end

    subgraph "Retrieval Runtime (retrieval.rs)"
        R --> B{"Backend Selection"}
        B -->|Default| JS["JournalSqliteFts"]
        B -->|Preview| EX["ExternalDerivedPreview"]

        JS --> F["Fusion Scoring"]
        EX --> F

        F --> S1["Lexical (FTS5)"]
        F --> S2["Vector (Embeddings)"]
        F --> S3["Recency / Quality"]
    end

    subgraph "Output"
        F --> H["MemorySearchHit"]
    end
```

Sources: [crates/palyra-daemon/src/retrieval.rs#1-10](http://crates/palyra-daemon/src/retrieval.rs#1-10), [crates/palyra-daemon/src/application/recall.rs#45-53](http://crates/palyra-daemon/src/application/recall.rs#45-53).

### Scoring Profiles

Different source kinds (e.g., `Memory` vs `Transcript`) use different scoring weights defined in `RetrievalSourceProfileKind` [crates/palyra-daemon/src/retrieval.rs#154-162](http://crates/palyra-daemon/src/retrieval.rs#154-162). For example, `Memory` hits often prioritize exact phrase matches and explicit trust labels [crates/palyra-daemon/src/application/recall.rs#138-156](http://crates/palyra-daemon/src/application/recall.rs#138-156).

### External Indexing

For high-scale environments, the daemon supports an `ExternalRetrievalRuntime`. This allows offloading vector search to external providers while maintaining the `JournalStore` as the absolute source of truth [crates/palyra-daemon/src/retrieval.rs#34-38](http://crates/palyra-daemon/src/retrieval.rs#34-38), [crates/palyra-daemon/src/retrieval.rs#103-113](http://crates/palyra-daemon/src/retrieval.rs#103-113).

Sources:

* `crates/palyra-daemon/src/application/memory.rs`
* `crates/palyra-daemon/src/application/tool_runtime/memory.rs`
* `crates/palyra-daemon/src/retrieval.rs`
* `crates/palyra-daemon/src/application/recall.rs`
* `crates/palyra-daemon/src/application/tool_registry/builtin.rs`
* `crates/palyra-cli/src/commands/memory.rs`