Retrieval-Augmented Generation (RAG) and Recall

The Palyra Retrieval-Augmented Generation (RAG) system provides agents with a high-fidelity mechanism to access durable principal memory, session-specific context, and workspace-scoped documents. It employs a hybrid retrieval strategy that combines lexical search, vector embeddings, and temporal recency to produce a ranked “Recall Plan”. This plan is then compiled into the model’s instruction set or injected into the turn context via the InstructionCompiler.

​

Retrieval Architecture and Hybrid Search

Palyra utilizes a multi-stage retrieval pipeline to resolve relevant context from the JournalStore. The system performs hybrid search by merging results from SQLite FTS5 (lexical) and vector similarity (semantic) indices crates/palyra-daemon/src/journal.rs#135-143.

​

Scoring Components

The final relevance score for a retrieval candidate is calculated using score_with_profile crates/palyra-daemon/src/gateway/runtime.rs#107-110, which aggregates:

1. Lexical Score: BM25-based matching via SQLite FTS5 virtual tables crates/palyra-daemon/src/journal.rs#138-140.
2. Vector Score: Cosine similarity against embeddings produced by a MemoryEmbeddingProvider crates/palyra-daemon/src/journal.rs#168-175.
3. Recency Score: A decay function that favors newer information based on created_at_unix_ms crates/palyra-daemon/src/retrieval.rs#106-107.

​

Retrieval Flow Diagram

This diagram illustrates the flow from a natural language query to the assembly of a RecallPlan within the GatewayRuntimeState. Natural Language to Retrieval Plan Flow Sources: crates/palyra-daemon/src/gateway/runtime.rs#105-110, crates/palyra-daemon/src/journal.rs#135-143, crates/palyra-daemon/src/application/tool_runtime/memory.rs#1-10

​

Recall Plan and External Indexing

The RecallPlan is the structured output of the retrieval process. It includes RetrievalBranchDiagnostics to provide transparency into why specific items were selected or filtered crates/palyra-daemon/src/journal.rs#81-82.

​

External Retrieval Index

For large-scale workspace documents, Palyra supports an external retrieval index. This is managed via the ExternalRetrievalRuntime crates/palyra-daemon/src/retrieval/external_index.rs.

• Chunking: Documents are split into segments based on WORKSPACE_CHUNK_TARGET_BYTES (default 1024) with a WORKSPACE_CHUNK_OVERLAP_BYTES (default 160) crates/palyra-daemon/src/journal.rs#156-157.
• Status Tracking: The WorkspaceRetrievalIndexStatus tracks the health and backfill progress of the vector index crates/palyra-daemon/src/journal/retrieval_index_status.rs#73-73.

​

Memory Scopes

Retrieval is strictly partitioned by MemoryLifecycleScope crates/palyra-daemon/src/application/memory.rs#42-42:

• principal: Global user-level facts and preferences.
• session: Context specific to the current conversation.
• channel: Platform-specific context (e.g., Discord server info).
• workspace/project: Filesystem-backed knowledge crates/palyra-daemon/src/application/tool_runtime/memory.rs#19-21.

Sources: crates/palyra-daemon/src/journal.rs#153-157, crates/palyra-daemon/src/application/tool_runtime/memory.rs#18-22, crates/palyra-daemon/src/gateway/runtime.rs#80-82

​

Instruction Compilation and Context Assembly

Once retrieval is complete, the InstructionCompiler assembles the final prompt context. This process ensures that retrieved information is presented to the model with appropriate “Trust Labels” and “Claim Boundaries” crates/palyra-daemon/src/application/instruction_compiler.rs#1-10.

​

Instruction Selection Logic

The InstructionCompiler generates CompiledInstructions based on:

1. Tool Catalog: Available tools for the turn crates/palyra-daemon/src/application/instruction_compiler.rs#61-61.
2. Trust Summary: Aggregated safety posture of the retrieved context blocks crates/palyra-daemon/src/application/instruction_compiler.rs#31-38.
3. Temporal Evidence: Current UTC/Unix timestamps to prevent model hallucination of dates crates/palyra-daemon/src/application/instruction_compiler.rs#149-150.

​

Context Assembly Diagram

This diagram shows how retrieved memory items are transformed into provider messages. Recall to Instruction Compiler Pipeline Sources: crates/palyra-daemon/src/application/instruction_compiler.rs#1-26, crates/palyra-daemon/src/application/instruction_compiler.rs#89-104, crates/palyra-daemon/src/application/tool_runtime/memory.rs#106-121

​

Key Implementation Details

​

Claim Boundaries

To prevent the model from making false assertions about missing information, Palyra uses “Claim Boundaries” crates/palyra-daemon/src/application/tool_runtime/memory.rs#12-16. For example:

• MEMORY_HITS_ABSENT_CLAIM_BOUNDARY: “no memory hits were returned; do not invent stored preferences or prior facts” crates/palyra-daemon/src/application/tool_runtime/memory.rs#85-86.
• SESSION_SEARCH_HITS_PRESENT_CLAIM_BOUNDARY: Instructs the model to cite hits as session recall, not durable memory crates/palyra-daemon/src/application/tool_runtime/memory.rs#98-98.

​

Memory Maintenance

The system performs background maintenance via the spawn_scheduler_loop crates/palyra-daemon/src/lib.rs#143-143. This includes:

• Embedding Backfill: Processing items missing vectors crates/palyra-daemon/src/journal.rs#77-78.
• Retention Enforcement: Purging expired memory items based on MEMORY_RETENTION_DAY_MS crates/palyra-daemon/src/journal.rs#150-150.

​

Tool Registry Integration

The palyra.memory.search and palyra.memory.recall tools are defined in the builtin tool registry with specific schemas for top_k, min_score, and scope crates/palyra-daemon/src/application/tool_registry/builtin.rs#105-124. Sources: crates/palyra-daemon/src/application/tool_registry/builtin.rs#105-135, crates/palyra-daemon/src/application/tool_runtime/memory.rs#1-22, crates/palyra-daemon/src/journal.rs#147-152