Module 1 · Foundations of Agentic AI · scripted

Exercise: The Window Lab

35 min
🎤 Talk view← Cockpit

The goal: feel the wall's edges

You've now seen both constraints — the window has edges, and every iteration re-bills the whole conversation. In this exercise you control both dials directly and watch what they do to an agent's ability to solve a problem. The instrument is the Window Lab: the Tool Playground with a context window you set.

How the window is enforced

You choose a window size in tokens.
Before every prompt is sent, the Lab walks backward from the newest message, keeping what fits — and EVICTS the older messages that don't.
One exception: the task message always survives. Everything else competes for the space that remains.

The exercise at a glance

  1. 1
    Step 1: run a task with a generous window and record the meters.
  2. 2
    Step 2: shrink the window until the agent breaks — and watch HOW it fails.
  3. 3
    Step 3: redesign tools, task, and results until the same task survives the small window.

The exercise playground

This exercise needs its playground — the Window Lab — because no chat product lets you set the context window yourself and watch the evictions happen. Everything from the Tool Playground is here (the tool format, templates, you-play-the-computer results), plus three new instruments.

How the playground works

  1. 1
    Open it: /exercises/window-lab — or scan the QR code on the next card.
  2. 2
    Read the meters as you run.
    Total input tokens across the run, and the same total WITH prefix caching (a synthetic 0.1× discount on tokens matching the previous prompt's prefix). Notice when the discount disappears: eviction rewrites the front of the prompt, and a rewritten prefix is a cache miss.
  3. 3
    Open "show prompts" to see every prompt actually sent.
    Each with its token count, its cached cost — and a ⚠ marker counting what was evicted.
  4. 4
    Use the ✨ buttons for synthetic results.
    A second LLM writes the tool result for you — short, medium, long, or overflow (deliberately larger than the space that remains) — so you can stress the window without typing forty lines yourself.

Scan to open the Window Lab

Scan to open the Window Lab
ai-agents-seminar.vercel.app/exercises/window-lab

Step 1 · Baseline

Step 1 · What you'll do

  1. 1
    Load a template — Filesystem explorer, or Search & read.
  2. 2
    Set the window generously: 8,000 tokens or more.
  3. 3
    Run the task to completion, playing the computer with ✨ medium results.
  4. 4
    Record the meters.

Capture — end of Step 1

💾 Save this before you move on

Copy into your course document and save:

  1. 1Window size, number of prompts, total tokens, cached total.
  2. 2From the prompt log: how much of prompt 5's cost was prefix — and what would this run cost without caching?

Step 2 · Shrink until it breaks

Step 2 · What you'll do

  1. 1
    Same task: halve the window and rerun. Then halve it again.
  2. 2
    Find the point where the agent stops being able to solve the problem.
  3. 3
    Watch HOW it fails — eviction failures have signatures.
    Re-reading a file it already read (the result was evicted) · contradicting an earlier finding · looping on a step it already did · forgetting what it concluded and starting over.
  4. 4
    Use ✨ long and overflow results to force evictions on demand.

Capture — end of Step 2

💾 Save this before you move on

Copy into your course document and save:

  1. 1The largest window at which the run FAILED, and the smallest at which it succeeded.
  2. 2The failure signature: which evicted message did the agent need, and what did it do without it? (The ⚠ markers in the prompt log show you exactly what was lost, and when.)
  3. 3What happened to the caching discount as evictions began?

Step 3 · Redesign to survive the small window

Now the real design challenge: go back to the window size that broke the agent — and make the task solvable there anyway, by redesigning what enters the conversation. Everything you control is on the table:

The levers

the TOOL DESCRIPTIONS — tighter, fewer tokens, still clear
the TASK — restated compactly; is every sentence earning
its place in the one message that always survives?
the RESULTS — you play the computer, so play a well-designed
one: return summaries instead of dumps, counts instead of
listings, a handle ("42 files — ask for a folder") instead
of contents
the TOOLS THEMSELVES — would a different tool need smaller
results? (a search tool instead of read-everything?)

Iterate until the same task completes inside the window that previously broke it.

Capture — end of Step 3

💾 Save this before you move on

Copy into your course document and save:

  1. 1Before/after: the failing design and the surviving one.
  2. 2The single change that bought the most room — and its cost in tokens saved per prompt.
  3. 3Final meters for the surviving run, with and without caching.

Before we regroup

💾 Save this before you move on

Copy your three captures into your course document and save — we compare together: the failure signatures people observed, the window sizes where tasks broke, the redesigns that survived, and the meters (the cheapest run in the room and the most expensive will differ by more than you expect).

Copy into your course document and save:

  1. 1All three captures above, in your course document.

Be ready to discuss:

  • The task message always survives eviction here. What ELSE, in your runs, deserved that protection — and what does that suggest about designing what goes where?
  • When evictions began, caching died with them. Say precisely why — and what that implies about how real systems should manage a full window.