Build a Simple Chatbot with Memory

Memory is just a list

A chatbot feels like it "remembers" your conversation, but an LLM is actually stateless — it forgets everything between calls. The trick behind every chatbot is delightfully simple: you keep a running list of messages and resend the whole thing each turn. The model re-reads the conversation every time and continues it. That's the entire secret to "memory."

The core loop

Here's a complete command-line chatbot in a dozen lines:

from openai import OpenAI
client = OpenAI()
history = [{"role": "system", "content": "You are a helpful tutor."}]

while True:
    user = input("You: ")
    if user.lower() in {"quit", "exit"}: break
    history.append({"role": "user", "content": user})
    resp = client.chat.completions.create(model="gpt-4o-mini", messages=history)
    reply = resp.choices[0].message.content
    history.append({"role": "assistant", "content": reply})
    print("Bot:", reply)

How it works

Trace one turn and the pattern is obvious:

1. The user types something; you append it to history as a user message.
2. You send the entire history to the model.
3. The model's reply is appended back as an assistant message.
4. Next turn, the growing history goes along too — so the bot "remembers" everything so far.

The system message at the top is sticky: it shapes the bot's behavior for the whole conversation.

Watch the context window

Resending everything has a catch: the conversation can't grow forever. Every model has a context window — a maximum amount of text it can consider at once — and a long chat will eventually exceed it. Two standard fixes:

• Keep a sliding window: retain the system message plus the last N turns, dropping the oldest.
• Summarize: once the history gets long, compress older turns into a single short note ("Earlier: the user is learning Python and prefers concise answers") and keep recent turns in full.

Common pitfalls

• Forgetting to append the reply — if you don't add the assistant message back to history, the bot has no memory of what it said.
• Letting history grow unbounded — costs rise with every token resent, and eventually you hit the limit.

The takeaway

A chatbot is a loop over a growing message list: append the user, send the history, append the reply, repeat. Add a strategy for trimming old turns and you've got the backbone of every conversational AI.

Try it now: Run the loop, then change the system message to give your bot a personality — a sarcastic chef, a patient math tutor — and notice how it stays in character across turns.