LEARNING OBJECTIVE
In this quick guide, you’ll learn about six memory systems that can be implemented in your LLM-powered applications. By understanding these systems, you’ll gain insights into how each serves a specific purpose in maintaining context, personalizing experiences, and improving task execution.
PRE-REQUISITES
You should have a basic understanding of Large Language Models (LLMs) and how they process input and output text. Familiarity with concepts like token limits, embeddings, and Retrieval-Augmented Generation (RAG) will help as well.
LET'S BEGIN!
[0]
MEMORY CATEGORIES
Short-Term Memory
Stores temporary information needed to maintain context during a single session.
Example: Remembering the last few messages in a chatbot conversation.
Working Memory
Manages task-specific information required to complete a process or step within a session.
Example: Keeping track of steps in a multi-turn workflow like code generation or form filling.
Long-Term Memory
Retains information persistently across multiple sessions for future reference.
Example: Remembering a user’s preferences or previous interactions over time.
User Profile Memory
Stores user-specific details to personalize responses and interactions. This is a subset of Long-Term Memory.
Example: Saving a user’s name, preferred topics, or goals to tailor future conversations.
Semantic Memory
Holds structured knowledge, facts, or domain-specific information that can be retrieved when needed.
Example: Answering FAQs or providing definitions from a knowledge base.
Procedural Memory
Stores predefined workflows or instructions to help the system perform specific tasks. Involves storing sequences of actions or workflows, aiding in task automation.
Example: Recalling step-by-step instructions for automating a process, like generating a report.
[1]
SESSION MEMORY SYSTEM
Handles temporary information for managing conversations and tasks within a single session.
Memory Categories Used:
Short-Term Memory
Working Memory
Uses:
Session Conversation
Persistence:
Temporary
Use Cases:
Keeping track of recent user messages (e.g., chat history).
Managing specific tasks like step-by-step workflows (e.g., code generation or form filling).
Implementation:
Sliding window for conversation history, task-specific buffers, and token-based context management.
[2]
PERSISTENT MEMORY SYSTEM
Retains user-specific information and historical context across sessions.
Memory Categories Used:
Long-Term memory.
User Profile memory.
Uses:
Multi-session Conversations
Persistence:
Persistent
Use Cases:
Remembering user preferences, names, or goals.
Retaining historical data to personalize interactions.
Implementation:
Store information in key-value databases or vector databases (e.g., Pinecone) for dynamic retrieval.
[3]
KNOWLEDGE MEMORY SYSTEM
Stores structured knowledge and workflows that the application can reference during interactions.
Memory Categories Used:
Semantic memory.
Procedural memory.
Uses:
Structured Data
Persistence:
Persistent
Use Cases:
Providing domain-specific answers (e.g., knowledge bases or FAQ bots).
Recalling pre-defined workflows or task instructions.
Implementation:
Use Retrieval-Augmented Generation (RAG) to fetch information from vectorized knowledge or structured JSON workflows.
[4]
EPISODIC MEMORY SYSTEM
Summarizes key interactions or events into compact, recallable episodes.
Memory Categories Used:
Long-term memory.
Uses:
Summarized Events
Persistence:
Persistent
Use Cases:
Providing recaps of past sessions or key moments.
Summarizing lengthy interactions for efficient retrieval.
Implementation:
Generate conversation summaries using the LLM and persist them in a database.
[5]
REFLEXIVE MEMORY SYSTEM
Dynamically adapts responses based on immediate feedback within a session.
Memory Categories Used:
Short-term memory.
Uses:
Real-time Adjustments
Persistence:
Temporary
Use Cases:
Refining responses based on user feedback (e.g., thumbs up/down).
Adjusting model behavior temporarily during the conversation.
Implementation:
Implement user feedback mechanisms to modify prompts or guide behavior in real time.
[6]
CONTEXTUAL MEMORY SYSTEM
Incorporates external inputs (e.g., APIs, location, time) into the current session.
Memory Categories Used:
Short-term memory.
Uses:
Dynamic Inputs
Persistence:
Temporary for real-time, session-specific inputs (e.g., current weather).
Persistent if external inputs are stored for reuse across sessions.
Use Cases:
Adding dynamic information like the current weather, time, or system states.
Making responses context-aware for external conditions.
Implementation:
Inject external variables or data from APIs into prompts.
RECAP
Most applications require a hybrid approach when it comes to memory implementation. Use short-term memory for maintaining context in current conversations. Add long-term memory for personalization and retention across sessions. Integrate semantic memory with RAG to bring in domain-specific knowledge and ensure accurate, dynamic responses. Consider episodic memory for summarization of interactions and user profile memory for personalization. By strategically combining these memory types and aligning them with your application’s goals, you can build an intelligent, adaptive, and scalable LLM app that not only meets your users’ needs but also grows with them over time.
NEXT STEPS
Start by identifying your application’s memory management needs. Explore vector databases for implementing long-term and semantic memory, and familiarize yourself with token limits to effectively manage short-term memory. Take a step-by-step approach to design a scalable and intelligent app that meets your goals.
Comments