Memory and context by hand

Already trimming history before every call? Skip to the next lesson.

Your agent works. Now make it affordable.

Remember from lesson 0.2 that you resend the whole conversation every time. That means a long chat gets slower and more expensive with every turn, and eventually stops working when it will not fit any more.

The idea

Context is a budget you spend, not a place things live.

The simplest fix is trimming: before each call, drop the messages you no longer need. A sensible first strategy is to keep the system message and the most recent turns, and drop the middle.

Keep the system message specifically. It holds the instructions. Drop it and your agent forgets how it is supposed to behave, which looks like the model going haywire when it is really your code throwing the rules away.

Later you can summarise instead of dropping, which keeps the meaning of old turns at a fraction of the size. Trimming first, because you can see it.

What to write

There is a long conversation and your Module 2 loop already in the file. Write trim_messages, then call it before every chat:

messages = trim_messages(messages, MAX_MESSAGES)

Watch the trace for context_trimmed. That event fires when your function actually drops something.

The point

Context is a budget you spend, not a place things live.

Break it on purpose

Set MAX_MESSAGES to 20 and run. Nothing gets dropped, because the conversation is shorter than the limit, and the check fails. A trimming function that never trims is not doing anything.