Σ 0:00▸ 0:00

Module 4 · Planning, Context & Multi-agent Systems · scripted

Thinking in the Loop: Channels for Thought

by Jules White

25 minoutcomes: agent-architectures, tool-design
🎤 Talk view← Course

Overview

The last lesson put a plan at the front of the conversation — and left an odd little tool unexplained: think( thought ), whose only reply is "OK." This lesson explains it, because planning does not stop when acting starts: the plan updates as evidence arrives, and that means the agent must be able to think in the middle of a conversation whose other words cause actions. Every design for this answers the same two questions — how does the model mark a thought as not-an-action, and how does the loop know when the speaking is over? We follow the answers through their history: labeled prefixes, the think tool, the done tool, and the native reasoning channels of modern models — and discover that once thought has a channel of its own, the loop can control it: force it, meter it, and budget it.

Two kinds of words

An agent's conversation carries two kinds of words, and everything in this lesson comes from needing to tell them apart:

Two kinds of words

words that acttool calls — spoken in the protocol, parsed and executed by the computer
words that thinkhypotheses, observations, updated plans — meant to be read, never executed

The computer must run the first kind, ignore the second, and know when neither is coming — when the task is done.

Recall the protocols lesson: the computer is not an intelligent reader — it can only parse what it expects. So thoughts and actions cannot simply be mixed in fluent prose. The conversation needs a convention that separates them, and the history of agent design is largely the history of that convention.

The oldest channel: prefixes

Before function calling, the separation was textual. The prompt taught a format in which every message carried a labeled thought and a labeled action:

Example 1 · Thought: and Action:
user

You have the following tools available for use:

  1. read_file( path )
  2. edit_file( path, old, new )
  3. run_tests( )

Task: The test suite is failing — find the problem and fix it.

Rules: Respond with exactly two lines each turn:

Thought: <your reasoning>

Action: <one tool call>

assistant

Thought: I should see what actually fails before touching code.

Action: run_tests()

user

2 failed: test_mean_skips_missing — expected 31.0, got -312.6…

assistant

Thought: The name says missing values should be skipped — the test file will state the rule.

Action: read_file( test_analysis.py )

user

[readings of -999.0 mark a missing value and must be excluded from the mean]

assistant

Thought: I understand the bug now. Let me just fix analysis.py directly — replace the plain mean with one that filters out -999.0.

user

ERROR: no "Action:" line found in your response. Could not parse.

✎ noteThought and action share one message, separated by nothing but a labeled line — a protocol living in the prompt. And it fails exactly the way the protocols lesson predicted: a probabilistic speaker eventually drifts from the format. The eager turn arrives without its "Action:" line, and the machine reader — which can only parse what it expects — breaks.

Function calling splits the words — and creates a fork

Function calling took the actions out of the text entirely: an action now travels as a structured tool call, unmistakable to the parser, trained into the model itself. Thoughts no longer need a label to be told apart from actions.

But this creates a new question where the old one used to be. Plain text is now unemployed — no action will ever travel in it — so the loop must decide what a plain-text message from the agent means. There are exactly two good answers:

The fork: what does plain text mean?

text means doneany plain-text message ends the run — it is read as the final answer. Thinking must then travel as a tool: think( thought ), which the computer answers with a flat "OK"
a done toolthe run ends only on an explicit call to done( summary ). Plain text between calls is now free — it IS the thinking channel

Take them one at a time, and start with the failure that motivates the first design.

Example 2 · Text means done — and thinking breaks it
user

The same tools as Example 1, now as real function calls. The loop's rule: any plain-text response ends the run — text is the final answer.

assistant
run_tests()
user
2 failed: test_mean_skips_missing — expected 31.0, got -312.6…
assistant

The test name suggests missing values should be skipped. I should read test_analysis.py next to confirm the expected behavior.

✎ noteA perfectly good thought — and the run is over. The loop read plain text, so this reflection became "the final answer," and the task ended half-finished. Under text-means-done, thinking out loud is fatal — unless thought gets a legal channel.
Example 3 · The think tool gives thought a channel
user

The same, plus one tool and one instruction:

  1. think( thought )

Never respond with plain text until the task is complete. To reason, call think. Roughly every other tool call, call think with your updated plan.

assistant
run_tests()
user
2 failed: test_mean_skips_missing — expected 31.0, got -312.6…
assistant
think(thought = "The name says missing values should be skipped.
Next: read the test to confirm, then make the smallest fix in
analysis.py.")
user
OK.
assistant
read_file(path = "test_analysis.py")
user
[readings of -999.0 must be excluded from the mean]
assistant
edit_file(path = "analysis.py",
  old = "sum(readings)/len(readings)",
  new = "mean of [r for r in readings if r != -999.0]")
user
OK.
assistant
run_tests()
user
All tests pass.
assistant

Fixed: mean_salinity now skips missing readings coded -999.0, and the full suite passes.

✎ noteThe thoughts travel as tool calls — visible in the conversation, answered by a flat "OK" whose only job is to keep the turns alternating. Plain text appears exactly once: at the end, where it means what the loop takes it to mean — done. This is the design every conversation in the planning lesson used. The odd little tool now has its reason.
Example 4 · The done tool sets the text free
user

The same tools — no think this time, but one addition:

  1. done( summary )

Think step by step between actions; plain text will not end the run. Call done when the task is complete.

assistant
run_tests()
user
2 failed: test_mean_skips_missing — expected 31.0, got -312.6…
assistant

The failing name points at missing-value handling. I'll read the test to confirm, then make the smallest fix.

user

[the loop, automatically] OK — continue.

assistant
read_file(path = "test_analysis.py")
user
[readings of -999.0 must be excluded from the mean]
assistant
edit_file(path = "analysis.py",
  old = "sum(readings)/len(readings)",
  new = "mean of [r for r in readings if r != -999.0]")
user
OK.
assistant
run_tests()
user
All tests pass.
assistant
done(summary = "mean_salinity now skips readings coded -999.0;
all tests pass.")
user
Run complete.
✎ noteNow the text between calls is the thinking channel — free, fluent, unparsed — and nothing ends until done is called. Notice who answered the thought: the conversation still takes turns, so the LOOP supplies the acknowledgment, automatically, and prompts again. Same "OK," different author.

The two designs are mirror images, and the symmetry is worth a card of its own:

Two designs, one OK

text means donethe model marks the thought — it calls think(...), and the computer answers OK
a done toolthe model just thinks in text — and the loop answers OK, until done() is called

Someone must always answer, because the conversation takes turns. The designs differ only in who marks the thought: the model, with a word — or the loop, with a rule.

The programmable mind

So why would anyone choose the think tool, with its stilted tool-call thoughts, over free-flowing text? Because a tool call is something the loop can see and govern. Once thinking is a word in the language, thinking becomes schedulable:

The loop can schedule thought

# every third iteration, offer ONLY the think tool:
if iteration % 3 == 0:
    tools_this_turn = [think]

# or gate on events:
if last_result.is_error or next_action.is_irreversible:
    tools_this_turn = [think]

The agent cannot act until it has reflected — not because the
system prompt asked nicely, but because this turn's language
contains no other word.

You know the mechanism from Module 2: the tool list bounds the action space. Here the loop narrows the language to a single word, and the only legal move is a soliloquy. "Roughly every other call, think" is a request; the tool gate is a law.

Example 5 · The forced soliloquy
assistant
edit_file(path = "analysis.py", old = "sum(readings)", new = "…")
user
ERROR: no match for "sum(readings)" — the file has changed since
it was last read.
[the loop: an error just came back — this turn, the only tool
offered is think( thought )]
assistant
think(thought = "The file changed under me — my map of it is
stale. Before editing again: re-read analysis.py, re-locate the
target line, and retry with fresh context.")
user
OK. [all tools restored]
✎ noteThe loop noticed the error and narrowed the language to one word. The agent could not lunge at a retry — the only legal move was reflection, and the reflection it wrote now conditions the retry that follows. This is planning in the moment, on a schedule the code controls: thoughtfulness, engineered into the conversation.

Reasoning models: the channel moves inside

Modern reasoning models ship with this machinery built in. They have a native reasoning channel — a separate stream, beside the conversation, where the model thinks before and between its visible outputs. It is the think tool's purpose absorbed into the model itself: thoughts get a lane of their own, marked not by a prefix or a tool call but by the architecture.

And the two dials we just built reappear in new clothes. The frequency dial — how often must it think — becomes the effort level: low, medium, or high, a budget of tokens devoted to the reasoning channel. And a dial we never asked for appears: visibility. Some vendors now hide the raw reasoning, because how a model thinks through agentic work has become core to its competitiveness — you receive a summary of the thinking, or nothing at all.

Then and now

think tool erathoughts are conversation: tool calls and labeled text, visible to everyone, scheduled by the loop's own code
reasoning modelsthoughts run in a native channel beside the conversation; the effort dial (low, medium, high) sets their token budget — and some vendors hide the channel entirely

Mark the cost of that last row. The previous lesson argued that a written plan is an inspectable artifact — the human can always see what the agent intended. A hidden reasoning channel is the same advantage inverted: the thinking happens, and helps, but nobody can read it. On the wall, we sometimes now get only the shadow of the thinking.

The map

The map of thinking channels

prefix protocollabeled lines — Thought and Action — inside one message; a protocol living in the prompt, and as fragile as any
think tooltext means done; thought travels as a tool call the loop can see, force, and meter
a done tooltext is free thinking; the run ends only on an explicit done( summary )
native channelthe model ships with its own reasoning lane; effort sets the budget; visibility varies

Four channels, one question: where do the thoughts go, and who controls when thinking happens?

The channel changed four times; the question never did. And notice what stayed constant underneath: thinking is still words — written into the conversation or alongside it, conditioning the trajectory exactly the way the plan did. Planning is not a phase that ends when acting begins. It is a running channel, and designing where it runs — and when it is forced to — is part of designing the agent.

Reading the channels
  • In Example 2, the run ended on a perfectly good thought. Under the done-tool design, what exactly would have happened to that same message?
  • After the forced soliloquy in Example 5, what does the conversation contain that makes the retry more likely to succeed? Point to the words.
  • When a vendor hides the reasoning channel, which advantage of write_plan from the last lesson is inverted — and what would you have your agent write down to win it back?
Your turn

Take the agent you forged in Module 2. (1) Decide: text-means-done with a think tool, or free text with a done tool — and defend the choice for your domain in two sentences. (2) Write the system-prompt sentence that sets its thinking cadence. (3) Name the one event in your domain that should FORCE a think before the next action — and write the loop's gating rule for it as pseudo-code.