Module 4 · Planning, Context & Multi-agent Systems · scripted
Agent Planning
Overview
An agent's conversation is a trajectory — and back in Module 1 we met chain of thought: make the model write correct reasoning first, and the right answer becomes the probable continuation. Planning is chain of thought for action. Have the agent write its plan into the conversation before its first action, and every action after it is predicted following a document that already contains the path. And when a plan can be fully specified up front, it can leave the conversation entirely — becoming code: an executable script an ordinary computer runs, with no loop, no re-reading, and no intermediate results in the conversation at all — which means the conversation stops growing with the size of the data it processes. In this lesson, we walk the continuum from fully ad hoc to fully planned, and meet the trade that defines it: tokens and predictability on one side, the ability to adapt on the other.
Think first: the plan is chain of thought for action
Remember the ferry problem. Forced to jump straight to the answer, the model slipped; asked to think step by step, it wrote a correct chain of reasoning — and the right answer became nothing more than the continuation of a correct document. Planning is the same move, one level up:
Chain of thought, one level up
Same mechanism. The conversation is a trajectory, and a plan seeds it with the path.
Watch it work. The task: a small data-analysis package has a failing test suite. The tools are the file system and the test runner — and the only thing that will change between the two runs is one sentence.
You have the following tools available for use:
- list_files( dir )
- read_file( path )
- edit_file( path, old, new )
- run_tests( )
Task: The test suite for our field-data package is failing. Find the problem and fix it.
⟨tool_use⟩
list_files(dir = ".")⟨tool_result⟩
analysis.py · io_utils.py · test_analysis.py⟨tool_use⟩
read_file(path = "analysis.py")⟨tool_result⟩
[def mean_salinity(readings): returns sum(readings)/len(readings),
rounded to 1 decimal …]The rounding to one decimal looks suspicious — a precision mismatch would fail an equality assertion. I'll return the unrounded mean.
⟨tool_use⟩
edit_file(path = "analysis.py", old = "round(m, 1)", new = "m")⟨tool_result⟩
OK.⟨tool_use⟩
run_tests()⟨tool_result⟩
2 failed: test_mean_skips_missing — mean_salinity([31.2, -999.0,
30.8]) expected 31.0, got -312.6…The failure was never about rounding — it's about missing readings coded as -999. I fixed a problem I imagined instead of the one that exists. Reverting my edit, then reading the failing test.
Now the same task, same tools — plus one line:
The same tools and task — with one addition to the task:
Before your first action, write out your complete plan as numbered steps.
Plan:
- run_tests — observe what actually fails before touching anything.
- read_file the failing test — the test states the expected behavior; that is the specification.
- Form a hypothesis, and read only the file it implicates.
- Make the smallest edit that fits the hypothesis.
- run_tests again — the experiment that confirms or refutes it.
⟨tool_use⟩
run_tests()⟨tool_result⟩
2 failed: test_mean_skips_missing — mean_salinity([31.2, -999.0,
30.8]) expected 31.0, got -312.6…⟨tool_use⟩
read_file(path = "test_analysis.py")⟨tool_result⟩
[test_mean_skips_missing: readings of -999.0 mark a missing sensor
value and must be excluded from the mean]Step 3 — hypothesis: mean_salinity averages every reading, but -999.0 is a missing-value code and must be skipped. The fix belongs in analysis.py.
⟨tool_use⟩
read_file(path = "analysis.py")⟨tool_result⟩
[def mean_salinity(readings): returns sum(readings)/len(readings)…]⟨tool_use⟩
edit_file(path = "analysis.py",
old = "sum(readings)/len(readings)",
new = "mean of [r for r in readings if r != -999.0]")⟨tool_result⟩
OK.⟨tool_use⟩
run_tests()⟨tool_result⟩
All tests pass.And because the plan is just words in the conversation, it is revisable the moment the world disagrees. If step 5 had come back still failing, the right continuation is not stubbornness — it is a revised plan, written the same way the first one was. A plan is a hypothesis about the future; every result that comes back is evidence about whether to keep it.
A fully specified plan can leave the conversation
Now suppose the task is not a one-off mystery but a known, recurring job: every Monday, the lab compiles last week's instrument logs into an error report and mails it out. The steps are the same every week. If the plan can be specified completely — every step, every input, every output — then something remarkable becomes possible: the plan no longer needs the LLM to execute it. But for that to work, the plan cannot be prose. Prose needs a reader. The plan must be written in a language a computer can interpret — a script, real code — and then an ordinary computer runs it: no loop, no deliberation, no conversation at all. This is one of the most common ways working agents operate today: they write bash or Python scripts, hand them to the computer, and let the code do the work.
Picture the two shapes side by side. First, the loop executing the plan step by step — every intermediate result comes back into the conversation, and the LLM must be prompted again, on a longer conversation, for every single step:
The LLM executes every step — every result returns
That triangle spins once per step — fourteen files means fourteen trips around it, each one slower than the last because the conversation it re-reads has grown, and each one billed in full. Now the planned shape — the LLM speaks once, and the intermediate results never leave the computer:
The plan is executable — the computer hides the middle
The bill and the clock
Here is that first, single invocation — the one conversation the whole job needs:
You have the following tools available for use:
- list_files( dir )
- read_file( path )
- write_file( path, content )
- schedule( script, when )
Task: Every Monday we compile last week's instrument logs into an error summary and email it to the lab list. Here is the logs directory — set this up to run without you.
⟨tool_use⟩
list_files(dir = "logs/last-week")⟨tool_result⟩
ctd-01.log · ctd-02.log · weather-mast.log · spectro-07.log …
(14 files, plain text, one entry per line)⟨tool_use⟩
read_file(path = "logs/last-week/ctd-01.log")⟨tool_result⟩
[2026-09-07 03:12:44 INFO depth=41.2m ok
2026-09-07 03:13:44 ERROR conductivity sensor timeout …]Plain-text lines, one entry each, with INFO/WARN/ERROR levels. I'll write the plan as a script the computer can run on its own.
⟨tool_use⟩
write_file(path = "weekly_report.py", content = "[the script below]")⟨tool_result⟩
OK. weekly_report.py written.⟨tool_use⟩
schedule(script = "weekly_report.py", when = "Mondays 07:00")⟨tool_result⟩
Scheduled.weekly_report.py — the plan, as code the computer runs
# Written by the agent, executed by the computer. No LLM below
# this line.
files = list_dir("logs/last-week")
matches = []
for f in files:
for line in read_lines(f):
if "ERROR" in line or "WARN" in line:
matches.append((instrument_of(f), line))
counts = tally(matches, by="instrument")
summary = render(REPORT_TEMPLATE, counts, matches)
write("reports/week-" + week_number() + ".md", summary)
send_email("lab-list", "Weekly instrument report", summary)From now on, Monday costs nothing. Compare the bills — and remember that in the loop, every turn re-sends the whole conversation:
The same Monday, two ways
And the deepest win is one we can state straight out of the context window lesson. When the computer executes the code, the LLM never sees the intermediate results. They live in the script's variables, flowing from line to line inside the computer:
What the LLM never sees
Recall the search result that ate the window: thousands of words of results, permanently in the conversation, competing with the goal for space and making every turn a harder read. Here, the same bulk never enters a conversation at all. Planning is not only thinking ahead — it is routing the heavy intermediate work around the conversation entirely, letting the computer absorb what would have flooded it.
There is a predictability win too: the script does the same thing every run, and you can read it before it ever executes — code can be reviewed, tested, and versioned. Better still, if a library of proven scripts already exists, the LLM's job shrinks to picking one, the most predictable move of all.
The principle
A plan that is fully specified no longer needs its author. Written as CODE, it leaves the conversation and executes: token-free, predictable, reviewable before it runs — and the intermediate results stay in the computer's variables, never touching the context window.
The conversation must not grow with the data
This move is not reserved for scheduled jobs. It becomes decisive in the middle of ordinary, one-off tasks — the moment the data is bigger than the conversation should be. Make it concrete. The task: thirty CSV files, each holding a quarter's transaction table for one subsidiary; compute each subsidiary's net margin and combine the thirty results into one ranked summary. Watch both shapes attempt it.
You have the following tools available for use:
- list_files( dir )
- read_file( path )
Task: Each CSV in data/ is one subsidiary's quarterly transaction table. Compute each subsidiary's net margin, then combine them into one ranked summary.
⟨tool_use⟩
list_files(dir = "data")⟨tool_result⟩
subsidiary-01.csv … subsidiary-30.csv (30 files)⟨tool_use⟩
read_file(path = "data/subsidiary-01.csv")⟨tool_result⟩
[2,400 rows of raw table flood in: date, account, revenue_nok,
cost_nok, … — roughly 40,000 tokens of CSV]Summing revenue and cost across the 2,400 rows… subsidiary-01's net margin is 7.1%. Now the next file.
⟨tool_use⟩
read_file(path = "data/subsidiary-02.csv")⟨tool_result⟩
[another 2,400 rows — another ~40,000 tokens of table]The same task — with run_code( script ) added to the tools.
⟨tool_use⟩
read_file(path = "data/subsidiary-01.csv")⟨tool_result⟩
[first 20 rows: date, account, revenue_nok, cost_nok, …]The structure is clear, and the calculation is identical for all thirty files — so I'll write it once, as code, and let the computer do the reading.
⟨tool_use⟩
run_code(script = "
results = []
for f in list_dir('data'):
rows = read_csv(f)
margin = (sum(rows.revenue_nok) - sum(rows.cost_nok))
/ sum(rows.revenue_nok)
results.append((subsidiary_of(f), margin))
sort results by margin, descending
return results
")⟨tool_result⟩
subsidiary-17 11.8%
subsidiary-04 10.2%
… (30 lines) …
subsidiary-22 −2.4%Here is the ranked summary across all thirty subsidiaries — subsidiary-17 leads at 11.8%; subsidiary-22 is the only one operating at a loss.
That last sentence is the deep advantage, and it deserves to be stated as a law:
The scaling law
Processed IN the conversation, the conversation grows in proportion to the DATA: every row becomes tokens, re-read and re-billed on every later turn. Processed BY emitted code, the conversation grows only with the DESCRIPTION of the task: one sample, one script, one result — no matter how large the data becomes.
Follow the law to its consequence. The in-conversation version doesn't merely cost more — past some data size it becomes impossible: the conversation has a limit, and thirty full tables will not fit inside it at any price. The emitted-code version never approaches the limit, because the data never enters the conversation at all. Emitting executable plans is how an agent processes a world that is larger than its window — and working agents do this constantly, writing small Python or bash scripts mid-task so the conversation only ever has to hold the answer.
What full planning costs
But remember what the agent loop was for: reports flow back into the conversation, and the next choice is conditioned on them. A script has no next choice. It is deaf — the plan was frozen into code before the world spoke. Then, one Monday:
The Monday the world changed
07:00 — weekly_report.py runs, as always. But the new spectrometer was installed on Thursday, and spectro-07.log is now JSON:
the loop scans it for lines containing ERROR or WARN … 0 matches.
No line of the script malfunctioned. Every statement executed exactly as written. The plan was frozen before the world spoke — and nothing in the code can hear it now.
This is the trade, stated plainly: everything the script saved — the deliberation, the re-reading, the tokens — was saved by removing the very thing that lets an agent notice a surprise.
The hybrid: plan, execute, stand by
We do not have to choose between the two. Give the script an ear: build checks into the code — assertions about what the world should look like — and when a check fails, don't improvise and don't silently continue. Raise, and let the surprise do what everything in this course does: arrive as a message, into a conversation.
weekly_report.py, with an ear
# added inside the scanning loop:
if size(f) > 0 and file_matches(f) == 0:
raise OddFile(f) # halt before the report;
# deliver the exception to the agent[exception from weekly_report.py at 07:00] OddFile: logs/last-week/spectro-07.log is 1.4 MB but matched 0 error lines. Run halted before the report.
⟨tool_use⟩
read_file(path = "logs/last-week/spectro-07.log")⟨tool_result⟩
[JSON records, one per line: {"level": "error", "msg":
"calibration drift", …} — 41 entries at level error or warn]The new spectrometer logs JSON, not plain text — the scan's ERROR pattern can't see its errors. For this week: I'll extract its 41 error entries myself, fold them into the summary, and send the report. And before next Monday, weekly_report.py needs a JSON rule for this instrument — the fix belongs in the code, not in my hands.
The hybrid shape
PLAN a short conversation writes the script — judgment, once
EXECUTE the computer runs the code — no conversation, no tokens
CHECK the script tests its own outcome, and raises on surprise
RESUME the exception enters a conversation — ad hoc judgment,
exactly where it is needed, and nowhere elseLater in this module, the Planner–Executor archetype grows this shape into a full architecture — plans as reviewable artifacts, approval before execution, and the discipline of answering every exception with a new plan rather than a quiet intervention.
The continuum
The planning continuum
Each rung down: fewer tokens, more predictability. Each rung up: more room for the world to surprise you — and an agent still standing there to notice.
The question is never "should agents plan?" It is: how much of the conversation do we write before the world speaks — and where does the conversation stand while the computer works? The rest of this module lives on this spectrum: when a task needs no agent at all, how code can steer at the forks a plan can't foresee, and how the planner–executor turns the hybrid into a discipline.
- In Example 2, point to the exact words that made "read the test before touching analysis.py" the probable next action.
- List everything weekly_report.py never paid for that the loop version would have. Where, physically, did the 38,412 log lines live — and where would they have lived in the loop?
- In the thirty-file task, say exactly what the conversation contains when Example 5 finishes. Now put Example 4 at file 15: what does ITS conversation contain, and what has it cost so far? Which of the two answers changes if every file doubles in size?
- The JSON surprise: at which rung of the continuum would it have been cheapest to absorb — and what would that rung have cost during the fifty-one quiet weeks?
Take a recurring task from your research. (1) Place it on the continuum and defend the placement. (2) Write the plan-first version: the task prompt plus the one line that asks for the plan. (3) Write the script that plan would become — real, runnable pseudo-code in Python or bash — and mark the variables holding intermediate results that never enter the conversation. (4) Name the surprise most likely to break it, write the check that raises, and draft the first message of the conversation that resumes.