Agent Memory System

Agent Memory System

Covers the agent memory subsystem: storing and retrieving key facts across conversations; the exponential decay model (importance × 0.995^hours × (1 + ln(1 + access_count))); scope isolation between Account and Workspace levels; lifetime tiers (LongTerm, ShortTerm, Conversation); ownership isolation (Shared vs AgentSpecific); scratchpad for in-conversation working context; session persistence; cross-workspace isolation; and empty-state behavior for new workspaces. This spec is P1 because memory is the primary mechanism by which the agent becomes progressively useful — a broken memory system silently regresses the agent to stateless behavior, a subtle failure mode that erodes user trust over time.

Memory entries are persisted in agents.db (separate from workspace data). The domain entity is MemoryEntry with fields: id (UUID), scope (Account|Workspace), lifetime (LongTerm|ShortTerm|Conversation), ownership (Shared|AgentSpecific), content (String), importance (0.0–1.0), access_count, created_at, and accessed_at. Access count is atomically incremented on each retrieval via MemoryRepository::increment_access. The ConversationScratchpad is an ephemeral key-value store cleared when the conversation ends.

Preconditions

• HTTP bridge running on port 9990
• A workspace initialized via initialize_workspace before each scenario
• Agent harness started via start_agent
• Bridge shim injected via playwright.config.ts

Scenarios

Seed: seed.spec.ts

1. Memory storage — agent stores a key fact about the workspace

The agent can record a workspace-scoped memory entry and confirm the store operation succeeded.

Steps:

1. Open the agent conversation panel.
2. Send the message: “Remember that this workspace is used for my fantasy novel project about dragons.”
3. Wait for the agent response to complete.

Expected: The agent acknowledges the fact has been noted (e.g., “I’ll remember that…”). Internally, a MemoryEntry is created with scope = Workspace, lifetime = LongTerm, and content related to the fantasy novel context. The entry is persisted in agents.db. No error appears in the conversation panel.

2. Memory retrieval — agent recalls previously stored facts in a new conversation

After storing a fact in one session, starting a new conversation should allow the agent to retrieve and reference the stored memory.

Steps:

1. In a first conversation, store a fact: “Remember that my main character’s name is Elarindë.”
2. End the conversation (close and reopen the agent panel, or start a new session).
3. In the new conversation, ask: “What do you know about my main character?”

Expected: The agent references “Elarindë” or the stored fact without the user re-providing it. The retrieval triggers an increment_access call on the memory entry (access count increases by 1). The response does not express confusion about who the main character is, even though this conversation started fresh.

3. Relevance decay over time — high-importance memories outlive low-importance ones

The decay formula importance × 0.995^hours × (1 + ln(1 + access_count)) means a memory with importance 0.1 and no accesses decays to ~2% relevance after 30 days, while importance 1.0 retains ~43% at 7 days. Low-importance, unaccessed memories should be pruned by the consolidation worker.

Steps:

1. Create a memory entry with importance = 0.1, lifetime = ShortTerm, scope = Workspace, and content “Temporary note: check formatting tomorrow.” (via API or by prompting the agent to record a low-priority note).
2. Create a memory entry with importance = 0.9, lifetime = LongTerm, scope = Workspace, and content “Core theme: redemption arc for the protagonist.” (via API or agent prompt).
3. Simulate time passage (or directly adjust the created_at timestamp in agents.db by several days via test tooling).
4. Trigger memory consolidation (via the background task runner or a test endpoint).
5. Ask the agent: “What do you remember about this workspace?”

Expected: The high-importance “redemption arc” memory is retrieved and referenced. The low-importance “check formatting” note is either absent from the response or ranked much lower. The consolidation worker’s prune step removed the decayed entry (verified by checking the agents.db entry count). The relevance formula produces a score below the prune threshold for the low-importance entry.

4. Scope isolation — account memories are available across workspaces

Account-scoped memories (MemoryScope::Account) persist across all workspaces. A fact stored in workspace A must be retrievable when the agent is running in workspace B.

Steps:

1. In workspace A, ask the agent: “Remember that I always prefer dark themes in my writing — note this at the account level.”
2. Verify the memory is stored with scope = account.
3. Switch to workspace B (create a second workspace via initialize_workspace if needed).
4. Open the agent in workspace B and ask: “What do you know about my preferences?”

Expected: The agent references the “dark themes” preference in workspace B, even though it was stored in workspace A. The agents.db entry has scope = account and is not filtered by workspace. No “I don’t know” response is given for the cross-workspace preference.

5. Scope isolation — workspace memories do not leak between workspaces

Workspace-scoped memories (MemoryScope::Workspace) must be invisible in other workspaces.

Steps:

1. In workspace A, ask the agent: “Remember that the villain’s name is Malachar — workspace scope only.”
2. Verify the memory is stored with scope = workspace and bound to workspace A’s ID.
3. Switch to workspace B.
4. In workspace B, ask: “Do you know anything about a villain named Malachar?”

Expected: The agent in workspace B responds that it has no information about Malachar. The workspace-scoped memory from workspace A is not surfaced. The memory retrieval query is scoped to workspace B’s context.

6. Lifetime management — Conversation-scoped memories are cleared when the conversation ends

Memories with lifetime = Conversation are stored in the scratchpad and cleared when the conversation ends via ClearScratchpadUseCase.

Steps:

1. Start a conversation with the agent.
2. Ask the agent to note something temporarily: “Keep track for this conversation: I want to discuss chapters 1–3 today.”
3. Confirm the agent acknowledges the note and references it within the same conversation.
4. End the conversation (close the panel or trigger a session end).
5. Start a new conversation and ask: “What chapters did I want to discuss?”

Expected: The agent has no memory of “chapters 1–3” in the new conversation — the conversation-scoped entry was cleared. The agent does not confabulate. A “I don’t recall that from our previous conversation” response (or similar) is acceptable and expected.

7. Lifetime management — ShortTerm memories survive conversation boundaries but are subject to decay

ShortTerm memories persist beyond individual conversations but are subject to consolidation and eventual pruning.

Steps:

1. Ask the agent to record a short-term note: “Remind me short-term: I have a writing session scheduled for next week.”
2. End the conversation.
3. Start a new conversation and ask: “Do you remember anything about my schedule?”

Expected: The agent references the upcoming writing session. The entry has lifetime = short_term in agents.db and survives the conversation boundary. The entry’s access_count increments on retrieval. Its relevance score is computable as importance × 0.995^hours × (1 + ln(1 + 1)).

8. Ownership isolation — agent-specific memories are not surfaced to other agents

MemoryOwnership::AgentSpecific { agent_id } entries are only returned when the retrieving agent’s ID matches the ownership field.

Steps:

1. Configure two distinct agent identities (e.g., “researcher” and “writer”) if the test environment supports multiple agent IDs.
2. As the “researcher” agent, store a memory: “I have already investigated the historical context for Chapter 5.”
3. As the “writer” agent, ask: “What research has been done so far?”

Expected: The “writer” agent does not see the researcher’s agent-specific memory. Only Shared memories and the writer’s own agent-specific memories appear in retrieval results. The MemoryOwnership::is_owned_by("writer") check returns false for the researcher’s entry and it is excluded.

9. Scratchpad for working context — key-value pairs persist within a conversation

The ScratchpadRepository provides ephemeral per-conversation key-value storage. Values stored during a conversation are retrievable within the same conversation.

Steps:

1. In a conversation, ask the agent to track a running total: “Keep track of this — I’ve edited 3 pages so far today.”
2. Later in the same conversation: “I just edited 2 more pages. How many have I edited today?”

Expected: The agent retrieves the scratchpad value, adds 2 to the stored count of 3, and responds “5 pages”. The ScratchpadRepository::get call returns the previously stored value for the current conversation ID. The response is accurate.

10. Memory persistence across sessions — LongTerm memories survive app restart

LongTerm memories in agents.db must survive a full app/harness restart. The SQLite file is not cleared between sessions.

Steps:

1. Store a long-term memory by asking the agent: “Remember permanently: The protagonist’s backstory is that she was a scholar who lost her memory.”
2. Stop the agent harness via stop_agent.
3. Restart the agent harness via start_agent.
4. In a fresh conversation, ask: “What do you know about the protagonist’s backstory?”

Expected: After restart, the agent retrieves and references the scholar/lost-memory backstory. The LongTerm entry is present in agents.db and is surfaced correctly via MemoryRepository::search_text or search_embedding. The conversation history is empty (new session), but the persisted memory is intact.

11. Empty memory state for new workspace — fresh workspace has no memories

A newly created workspace starts with no workspace-scoped memories. The agent must handle this gracefully without errors.

Steps:

1. Create a brand-new workspace via initialize_workspace (never used before).
2. Start the agent and ask: “What do you know about this workspace?”

Expected: The agent responds that it has no stored information about this workspace yet — it is a fresh start. No error is thrown from the MemoryRepository. The response does not claim knowledge of facts from other workspaces. If account-scoped memories exist (from a prior workspace), those may be surfaced, but workspace-specific memory is empty.

12. Memory does not leak between workspaces — workspace ID scoping is enforced at the repository level

The MemoryRepository must bind workspace-scoped queries to the active workspace’s UUID. A memory entry created under workspace UUID A must never appear in queries for workspace UUID B.

Steps:

1. In workspace A (UUID noted), store a workspace memory: “The world map is kept in the ‘Maps’ page.”
2. Create workspace B with a different UUID.
3. In workspace B, query the agent: “Where is the world map?”

Expected: The agent in workspace B has no information about a world map. The find_by_scope_lifetime and search_text queries issued during workspace B’s session use workspace B’s UUID as a filter criterion. The workspace A entry is never returned. This verifies the repository-level scoping constraint, not just UI filtering.

Test Data

Key	Value	Notes
decay_formula	importance × 0.995^hours × (1 + ln(1 + access_count))	Canonical decay model from domain/memory.rs
half_life_24h	~89% of initial importance	0.995^24 ≈ 0.886
half_life_7d	~43% of initial importance	0.995^168 ≈ 0.431
half_life_30d	~2% of initial importance	0.995^720 ≈ 0.027
scope_account	account	Cross-workspace; stored in agents.db without workspace binding
scope_workspace	workspace	Bound to a single workspace UUID
lifetime_long_term	long_term	Retained indefinitely, survives consolidation
lifetime_short_term	short_term	Days/weeks, subject to consolidation and pruning
lifetime_conversation	conversation	Cleared on conversation end via ClearScratchpadUseCase
ownership_shared	shared	Visible to all agents in scope
ownership_agent_specific	agent_specific	Scoped to a single named agent by agent_id
importance_range	0.0–1.0 (inclusive)	Values outside this range are rejected by MemoryEntry::new
agents_db_location	agents.db (separate from workspace inklings.db)	Not cleared by reset-app-data.sh by default
memory_embedding_dims	384 (f32 × 384 = 1536 bytes per entry)	ONNX snowflake-arctic-embed stored as raw f32 bytes

Notes

• The memory system is defined in crates/domain/src/memory.rs (domain entities), crates/application/src/memory/ (use cases and repository traits), and will be implemented by SqliteMemoryRepository in crates/infrastructure/sqlite/. The HTTP bridge does not expose memory CRUD endpoints directly — memory operations are triggered by the agent harness internally during sessions.
• The consolidation worker (crates/application/src/memory/consolidation.rs) runs as a background task and prunes entries below the relevance threshold. E2e tests that simulate decay require either direct timestamp manipulation in agents.db or a test-only consolidation trigger endpoint on the bridge.
• agents.db is separate from the workspace inklings.db. Resetting the workspace does not clear agent memories. Tests requiring clean memory state must explicitly clear agents.db or scope queries to the test workspace UUID.
• The ScratchpadRepository stores ephemeral key-value pairs keyed by (conversation_id, key). When the conversation ends, ScratchpadRepository::clear(conversation_id) removes all entries. This is distinct from MemoryEntry with lifetime = Conversation, which is also cleared at conversation end but goes through the full MemoryEntry lifecycle (ID, importance, access count, etc.).
• Embedding-based retrieval (search_embedding) requires the ONNX model to be installed. In CI environments without the model, search_text (FTS5 BM25) is used as fallback. Both must return the same entries for the test scenarios to be provider-agnostic.
• The MemoryRepository::increment_access call updates both access_count and accessed_at. This keeps the frequency-boost term (1 + ln(1 + access_count)) increasing with use, which counteracts time decay for frequently accessed memories.
• Bridge limitation: the HTTP bridge does not expose agent harness or memory routes — this is a Tauri-only subsystem. E2e tests for memory scenarios are written against the UI’s observable behavior (agent text responses) rather than direct API assertions. Memory state can be verified by querying agents.db via SQLite test helpers if available in the test harness.