Hermes Agent Memory and Self-Improvement

How Hermes Memory Works

Hermes Agent's memory system is fundamentally different from the conversation history you find in typical chatbots. While most AI assistants maintain a rolling window of recent messages that gets cleared between sessions, Hermes builds a persistent, structured knowledge base that grows over the entire lifetime of the agent. This memory is agent-curated, meaning the agent itself decides what information is worth retaining based on relevance, frequency of use, and importance to ongoing projects.

The memory system uses SQLite as its storage backend, with FTS5 (Full-Text Search version 5) providing fast keyword-based retrieval across the entire memory corpus. When the agent processes a new message, it queries the memory database for relevant context before constructing its response. This means the agent can recall details from conversations that happened days, weeks, or even months ago, as long as it determined those details were worth remembering.

Memory is organized into several categories that serve different purposes. Project memories store details about what you are working on, including goals, progress, blockers, and decisions made. User preferences track how you like things done, your communication style, your technical skill level, and your stated priorities. Session history maintains a searchable log of past interactions with LLM-summarized recall. Relational context, powered by Honcho dialectic user modeling, captures the dynamics of your working relationship with the agent.

Agent-Curated Memory with Periodic Nudges

The phrase "agent-curated memory with periodic nudges" describes the memory management strategy. The agent does not store everything. It actively filters incoming information and decides what is worth keeping based on learned patterns of usefulness.

The curation process works in two phases. During active conversations, the agent tags information with relevance scores based on factors like whether it relates to an ongoing project, whether the user explicitly emphasized it, or whether it contradicts or updates previously stored information. Low-relevance information (greetings, small talk, repetitive confirmations) is typically not persisted.

The "periodic nudges" component refers to a background process that periodically reviews the memory database and performs maintenance. This includes consolidating redundant memories, updating outdated information, adjusting relevance scores based on usage patterns, and flagging memories that may need user confirmation. For example, if you told the agent three months ago that you were using Python 3.10, but recent conversations show you consistently reference Python 3.12 features, the agent may nudge you to confirm the update.

FTS5 Cross-Session Recall

SQLite FTS5 is the search engine powering Hermes's memory retrieval. FTS5 provides ranked full-text search with support for phrase queries, boolean operators, and prefix matching. This means the agent can quickly find relevant memories even in databases containing thousands of entries.

When a new task arrives, the agent constructs a search query from the key terms in the user's message and retrieves the top-ranked matching memories. These memories are included in the context sent to the language model, giving it access to historical information without requiring the entire conversation history to be loaded. This approach is both faster and more token-efficient than sending full transcripts.

The search process also considers temporal relevance. Recent memories are weighted slightly higher than older ones, though this bias can be overridden for long-term project information that remains relevant indefinitely.

Honcho Dialectic User Modeling

Honcho is the relational intelligence layer in Hermes's memory system. While standard memory stores facts, Honcho models the dynamics of the user-agent relationship. It tracks patterns in how you communicate, what topics you care about, when you tend to be satisfied or frustrated, and how your preferences evolve over time.

The dialectic modeling approach means Honcho does not just record preferences statically. It models the interaction as an ongoing dialogue where both parties influence each other. If you consistently correct the agent's formatting choices, Honcho adjusts the agent's formatting behavior without requiring an explicit preference statement. If you express frustration with verbose responses, the agent learns to be more concise with you specifically.

This user-specific adaptation happens at the memory layer, not the prompt layer. The agent's soul file defines its general personality and behavior, but Honcho fine-tunes the execution based on individual user patterns. In multi-user environments, each user gets a personalized interaction experience shaped by their own Honcho profile.

Memory and Self-Improvement Integration

The memory system is tightly integrated with the self-improvement pillar, which governs when the agent reflects on its performance and how it updates its knowledge base. During reflective phases, the agent reviews recent interactions and evaluates the quality of its memory retrieval. Did it recall the right context? Were important memories missing? Did irrelevant memories clutter the response?

Based on this evaluation, the self-improvement system may create new memories, adjust relevance scores, consolidate related memories into more comprehensive entries, or flag gaps in the knowledge base. This continuous refinement means the memory system becomes increasingly accurate and useful over time.

Privacy and Data Control

All memory data is stored locally in the agent's SQLite database. Nothing is transmitted to external servers unless you explicitly configure an external backup or synchronization service. You can inspect the memory database at any time using standard SQLite tools, and you can delete specific memories or entire categories if needed.

Hermes also supports memory export and import, allowing you to migrate your agent's knowledge base to a new installation or share sanitized memory snapshots with team members. The export format is documented and human-readable, ensuring transparency about what data the agent has stored.

Memory Capacity and Scaling

The SQLite-based memory system handles typical personal and small-team workloads without performance concerns. A single user interacting with the agent 50 to 100 times per day accumulates roughly 10,000 to 30,000 memory entries per year, which SQLite handles efficiently. FTS5 search queries against databases of this size return results in under 10 milliseconds on modest hardware, meaning memory retrieval adds negligible latency to the agent's response time.

For deployments that accumulate hundreds of thousands of memory entries (heavy multi-user setups running for extended periods), FTS5 search performance may degrade gradually. The periodic memory curation process mitigates this by pruning low-value entries and consolidating related memories, keeping the active database manageable. Users who need to scale beyond SQLite's practical limits can implement a custom memory backend using PostgreSQL with pgvector or a dedicated vector database, though this requires plugin development and is not a common need.

Practical Memory Management Tips

Several practices help you get the most value from the memory system. First, explicitly tell the agent when information is important for long-term retention. Phrases like "remember that our production database is on port 5433" signal high importance and increase the likelihood of persistent storage. Second, correct the agent when it recalls outdated information, which triggers an update to the stored memory rather than creating a conflicting entry.

Third, use the memory inspection tools periodically to review what the agent has stored about you and your projects. The web dashboard provides a searchable memory browser, and you can also query the SQLite database directly for more advanced searches. This transparency helps you understand why the agent makes certain assumptions and allows you to correct inaccuracies before they compound over time.

Fourth, configure the memory search result count based on your model's context window size. The default setting loads the top 10 matching memories into context for each interaction. If you are using a model with a large context window (128K+ tokens), increasing this to 20 or 30 results provides richer context at the cost of additional token consumption. If you are using a smaller or cheaper model, reducing it to 5 results keeps costs down while still providing the most relevant historical context.