Skip to main content
The Memory and Learning subsystems in Palyra provide the agent with long-term persistence, factual recall, and behavioral adaptation. This is achieved through a hybrid storage approach combining vector-backed semantic search, FTS5-powered lexical search, and a “Learning” pipeline that extracts durable facts and preferences from conversation history.

​
Memory Architecture and Vector Store

Palyra utilizes a SQLite-backed JournalStore to manage memory items. Each memory item consists of raw text, metadata (tags, source, confidence), and an embedding vector for semantic retrieval [crates/palyra-daemon/src/journal.rs#64-100](http://crates/palyra-daemon/src/journal.rs#64-100).

​
Implementation Details

  • Storage: Memory is stored in the memory_items table within the SQLite journal.
  • Search: The system performs hybrid search by combining:
    1. Vector Search: Uses cosine similarity against embeddings generated by a MemoryEmbeddingProvider [crates/palyra-daemon/src/journal.rs#64-68](http://crates/palyra-daemon/src/journal.rs#64-68).
    2. Lexical Search: Integrated via SQLite FTS5 for exact keyword matching.
    3. Recency: Scoring is weighted by the created_at_unix_ms timestamp to prefer fresher information [crates/palyra-cli/src/commands/memory.rs#131-139](http://crates/palyra-cli/src/commands/memory.rs#131-139).
  • Embeddings: By default, the system uses a HashMemoryEmbeddingProvider (64 dimensions) for local, deterministic testing, but supports pluggable models [crates/palyra-daemon/src/journal.rs#71-79](http://crates/palyra-daemon/src/journal.rs#71-79).

​
Data Flow: Memory Injection

When a run is initialized, the prepare_model_provider_input function automatically queries the memory store based on the current session context and injects relevant snippets into the LLM prompt [crates/palyra-daemon/src/application/run_stream/orchestration.rs#16-18](http://crates/palyra-daemon/src/application/run_stream/orchestration.rs#16-18).

​
The Learning Subsystem

The Learning subsystem is a background process that reflects on completed conversations to extract “Durable Facts” and “User Preferences” [crates/palyra-daemon/src/gateway/runtime.rs#85-97](http://crates/palyra-daemon/src/gateway/runtime.rs#85-97).

​
Learning Pipeline

  1. Post-Run Reflection: After a run completes, a REFLECTION_TASK_KIND background task is scheduled [crates/palyra-daemon/src/background_queue.rs#11-12](http://crates/palyra-daemon/src/background_queue.rs#11-12).
  2. Candidate Extraction: The system analyzes the transcript to identify potential new facts or preferences, assigning them a Confidence Score in basis points (bps) [crates/palyra-daemon/src/gateway/runtime.rs#151-166](http://crates/palyra-daemon/src/gateway/runtime.rs#151-166).
  3. Thresholds:
    • Auto-Write: If confidence exceeds durable_fact_auto_write_threshold_bps (default 9000), the fact is committed to memory immediately [crates/palyra-daemon/src/gateway/runtime.rs#160](http://crates/palyra-daemon/src/gateway/runtime.rs#160).
    • Review: If confidence is lower but above the review_min_confidence_bps, it is placed in a “Learning Candidate” state for manual operator approval [crates/palyra-daemon/src/gateway/runtime.rs#159-161](http://crates/palyra-daemon/src/gateway/runtime.rs#159-161).

​
Natural Language to Code Entity Mapping: Learning Flow

This diagram shows how a user message results in a durable code entity in the database. Title: Learning Extraction Pipeline Sources: [crates/palyra-daemon/src/background_queue.rs#11-15](http://crates/palyra-daemon/src/background_queue.rs#11-15), [crates/palyra-daemon/src/gateway/runtime.rs#8-27](http://crates/palyra-daemon/src/gateway/runtime.rs#8-27), [crates/palyra-daemon/src/journal.rs#64-71](http://crates/palyra-daemon/src/journal.rs#64-71)

​
Memory Maintenance and Retention

Memory is not infinite; it is managed by a background loop that enforces MemoryRetentionPolicy [crates/palyra-daemon/src/journal.rs#64-66](http://crates/palyra-daemon/src/journal.rs#64-66).

​
Maintenance Loop

The spawn_background_queue_loop handles periodic maintenance tasks [crates/palyra-daemon/src/background_queue.rs#31-35](http://crates/palyra-daemon/src/background_queue.rs#31-35):
  • TTL Expiry: Items with a ttl_unix_ms in the past are purged.
  • Size Constraints: If retention_max_bytes or retention_max_entries are exceeded, the system evicts the oldest/lowest confidence items [crates/palyra-daemon/src/gateway/runtime.rs#78-80](http://crates/palyra-daemon/src/gateway/runtime.rs#78-80).
  • Embedding Backfill: A background process scans for memory items missing vectors and generates them using the configured provider [crates/palyra-daemon/src/transport/http/handlers/console/memory.rs#143-147](http://crates/palyra-daemon/src/transport/http/handlers/console/memory.rs#143-147).

​
Memory Management Components

Title: Memory System Class Interactions Sources: [crates/palyra-daemon/src/journal.rs#64-100](http://crates/palyra-daemon/src/journal.rs#64-100), [crates/palyra-daemon/src/gateway/runtime.rs#72-82](http://crates/palyra-daemon/src/gateway/runtime.rs#72-82), [crates/palyra-daemon/src/transport/http/handlers/console/memory.rs#11-22](http://crates/palyra-daemon/src/transport/http/handlers/console/memory.rs#11-22)

​
Configuration and Limits

Memory behavior is governed by FileMemoryConfig in the daemon configuration [crates/palyra-common/src/daemon_config_schema.rs#183-190](http://crates/palyra-common/src/daemon_config_schema.rs#183-190).
Constant / ConfigDefault ValueDescription
MAX_MEMORY_ITEM_BYTES16 KBMaximum size of a single memory string [crates/palyra-daemon/src/gateway.rs#118](http://crates/palyra-daemon/src/gateway.rs#118).
MAX_MEMORY_SEARCH_TOP_K64Maximum number of hits returned per query [crates/palyra-daemon/src/gateway.rs#117](http://crates/palyra-daemon/src/gateway.rs#117).
auto_inject_max_items3Number of memory snippets injected into LLM context [crates/palyra-daemon/src/gateway/runtime.rs#141](http://crates/palyra-daemon/src/gateway/runtime.rs#141).
retention_vacuum_schedule0 0 * * 0Cron expression for database vacuum (Weekly) [crates/palyra-daemon/src/gateway/runtime.rs#146](http://crates/palyra-daemon/src/gateway/runtime.rs#146).

​
API and CLI Interface

Operators can manage knowledge via the palyra memory command group [crates/palyra-cli/src/commands/memory.rs#4-14](http://crates/palyra-cli/src/commands/memory.rs#4-14).
  • Search: palyra memory search "query" triggers MemoryServiceClient::search_memory [crates/palyra-cli/src/commands/memory.rs#69-72](http://crates/palyra-cli/src/commands/memory.rs#69-72).
  • Recall: Simulates the auto-injection logic to show what the agent “sees” for a given prompt.
  • Index: Manually triggers the embeddings backfill loop [crates/palyra-daemon/src/transport/http/handlers/console/memory.rs#143-150](http://crates/palyra-daemon/src/transport/http/handlers/console/memory.rs#143-150).
Sources:
  • crates/palyra-daemon/src/journal.rs
  • crates/palyra-daemon/src/gateway.rs
  • crates/palyra-daemon/src/gateway/runtime.rs
  • crates/palyra-daemon/src/background_queue.rs
  • crates/palyra-daemon/src/transport/http/handlers/console/memory.rs
  • crates/palyra-cli/src/commands/memory.rs
  • crates/palyra-common/src/daemon_config_schema.rs