The Context Window: A Limit on the Conversation

Overview

There is a limit to how long a conversation the LLM can read at one time. That limit — the context window — makes space in the conversation a scarce resource, and the goal, the tools, and every result compete for it. And well before the hard limit, length itself costs us: the longer the conversation gets, the harder it is for the LLM to work with — just as your own reading comprehension strains as a text grows longer. In this lesson, we look at what the window is, how one careless tool result can flood the entire conversation, and how a well-designed result respects the space it shares.

The conversation has a limit

An LLM cannot read an endless conversation. There is a hard limit on how long a conversation it can take in at once — and everything we want it to consider must fit inside that limit. We cannot put the entire world into the conversation; if we could, the LLM would probably perform a great deal better. But there is only so much conversation it can read.

That limit is called the context window.

A page limit

Another way to think about it: the LLM has a page limit on the length of the conversation it can read and deal with at once. Everything we have built — the script, the trajectory, the tools, the results — must fit inside that limit, and we have to use it very, very efficiently.

The length of the conversation is measured in tokens. We will look at tokens properly later; for now, think of a token as roughly a word. So the question "how big is the window?" is roughly: how many words can it read at one time?

And this is a hard boundary on agents: the agent's ability to work is inherently constrained by the length of the conversation we can create and have it reason about. An agent is a conversation — so the window is the size of the agent's world.

Longer is harder — even before the limit

The window is the hard edge, but the pressure starts long before you hit it. Think about your own reading: a paragraph, you hold effortlessly; a chapter, you follow with some work; a four-hour transcript, and you start losing the thread — details blur, and the important sentence on page 12 competes with everything around it. The LLM is the same kind of reader. The longer the conversation grows, the harder it becomes for the LLM to use it well — the crucial detail gets buried, and comprehension degrades. So a short, dense conversation is not just cheaper than a long, rambling one. It is easier to read — and the agent reasons better because of it.

Everything competes for the same space

The window creates two pressures, and they squeeze from both ends.

First: the amount of information we can give the LLM to understand the situation, the goal, and the task is limited by the window.

Second: as the agent takes actions and receives information back about the external world, those results are limited by the same window — and here is the crucial part — everything is competing for the same space:

So watch what happens when one participant is careless with the shared space:

⟨tool_use⟩
search_email(query="Molde")

⟨tool_result⟩
[Returning all matching content]
Email 12, full text, 1,400 words: "Hotel Alexandra — booking
confirmed… (terms and conditions, cancellation policy, loyalty
program, parking information, breakfast hours…)"
Email 47, full text, 2,100 words: "SAS schedule change… (fare rules,
baggage allowances, codeshare notices, lounge access…)"
Email 51, full thread, 3,800 words: "Fwd: fjord kayaking?" (six
forwarded replies, signatures, and disclaimers)

⟨tool_use⟩
search_email(query="Molde")

⟨tool_result⟩
[12] Hotel Alexandra — "Booking confirmed" — reservation unchanged
[47] SAS — "Schedule change" — flight WF562 24 Aug now departs 14:05
[51] Erik — "Fwd: fjord kayaking?" — proposes Saturday morning

⟨tool_use⟩
read_email(id=47)

⟨tool_result⟩
"Your SAS flight WF562 Oslo–Molde on 24 Aug now departs 14:05
(previously 11:20)."

Spending the space well

The context window turns conversation design into an engineering discipline. A good tool result is not only accurate — it is economical: for every piece of the world, we must ask what the LLM actually needs in the conversation to make its next decision, and what is merely occupying space and making the whole thing harder to read. The goal statement, each tool description, each result a tool sends back — every one of them is a choice about how to spend a limited budget.

Much of what comes later in this course — summarizing, retrieving, revealing information in stages — is a set of strategies for exactly this problem. For now, carry the constraint itself: the conversation has a limit, and everything in it must earn its place.