Memory in AI Applications

Memory in AI Applications

Language models have no memory by default. Each conversation starts from scratch. But real applications need continuity — a customer support bot should remember the ticket you opened yesterday, a personal assistant should know your preferences.

Building memory into AI applications is a design problem, not a model capability. Here's how it's done.

The Four Types of Memory

AI application developers typically think about four kinds of memory:

1. In-context memory (working memory) The conversation history in the current context window. Everything the model can "see" right now. Limited by context window size and cost.

2. External storage (long-term memory) A database of facts, preferences, and history. Retrieved and injected into context when relevant — essentially RAG over your own conversation history.

3. Semantic memory (knowledge) A vector store of facts and documents the agent can look up. RAG over external knowledge.

4. Episodic memory (experiences) A record of past interactions and their outcomes, used to guide future behaviour. More complex to implement.

How Applications Build Memory

Short conversations: just keep the full message history in the context window. Most chat apps do this.

Long conversations: summarise older sections before they're pushed out of the window. Keep a rolling summary that captures key facts.

Cross-session memory: after each conversation, extract key facts ("user prefers formal tone", "user is building a React app", "user's name is Rich") and store them. Retrieve relevant facts at the start of each new conversation and inject them into the system prompt.

Vector-search memory: embed every turn of every conversation. When a new conversation starts, find past turns that are semantically similar to the current topic and inject them.

ChatGPT's "memory" feature and Claude's memory feature both work roughly like the cross-session pattern — key facts are stored and retrieved per user.

Memory and Privacy

Memory systems raise real privacy questions:

• What gets stored, and for how long?
• Can the user see and delete what's remembered?
• Is memory shared across users (should never be) or isolated per user?
• Is sensitive information (medical details, financial data) stored?

Good memory implementations are transparent — users can see what's remembered, edit it, and delete it. They're also scoped: one user's memory never leaks to another.

When building memory features, treat stored memories as sensitive user data. Apply the same data protection standards you'd apply to any personal information in your application.