Perception, Memory, and Context
Seeing and remembering
Perception
Perception means how the agent receives information. In a text-based system, perception may be the user's message, tool output, or documents. In a richer system, it could also include images, audio, sensors, or logs.
Memory types
Agents may use short-term memory for the current interaction and longer-term memory for useful facts, preferences, or past outcomes. Good memory helps an agent stay consistent and avoid repeating work.
Short-term vs long-term memory
Short-term memory
- Holds current context
- Useful for immediate reasoning
- Often limited in size
Long-term memory
- Stores durable information
- Useful across sessions
- May require retrieval methods
Memory is not always perfect
Agents can remember the wrong thing, remember too much, or retrieve irrelevant information. Memory must be managed carefully.
What is one purpose of memory in an AI agent?
Memory lets the agent reuse information across steps.
Correct answer: To store useful information for later
What is one example of information an agent might store in long-term memory?
Long-term memory often stores persistent preferences or facts.
Correct answer: A user's preferred writing style
Context window
For many language-model agents, the context window is the amount of text the model can consider at once. If information falls outside that window, the agent may need retrieval or memory systems to access it again.
MEMORY & CONTEXT
How agents hold on to—and retrieve—what they know
Types of Agent Memory
| Type | Scope | Storage | Example |
|---|---|---|---|
| In-Context | Current session | LLM token window | Conversation history |
| External / Vector | Persistent | Vector DB (Pinecone) | Knowledge base |
| Episodic | Past runs | Structured DB | "Last time I did X…" |
| Procedural | Always available | System prompt | Built-in skills |