Module 3 · Agent Tools, Knowledge, Memory & Research Design · scripted

Tool Design Case Studies

35 minoutcomes: tool-design

Four cases, one method

Everything so far — names as pointers, descriptions as paid definitions, the tool set as a language — becomes concrete the same way: take one goal, design two tool sets for it, and watch what happens to the trajectory. Four cases. Each ends in a principle you can reuse.

Case 1 · Editing a book

The goal: revise a 300-page manuscript. Two languages for it:

Two tool sets for one manuscript

Design A · positionalDesign B · structural
read_page(n)read_toc()
edit_page(n, line, text)read_section(id)
insert_on_page(n, line, text)edit_section(id, old, new)
delete_from_page(n, line)append_to_section(id, text)
Example 1a · The page numbers move
user

Delete the duplicated paragraph in chapter 2, then fix the typo "Mlde" in the chapter about the fjord crossing — I saw it around page 87.

assistant
⟨tool_use⟩
read_page(n = 41)
user
⟨tool_result⟩
[page 41 text: …the duplicated paragraph appears here…]
assistant
⟨tool_use⟩
delete_from_page(n = 41, line = 12)
user
⟨tool_result⟩
OK. Document is now 299 pages.
NoteQuietly, the world just shifted. Everything after page 41 moved up — but the conversation still says the typo is "around page 87."
assistant
⟨tool_use⟩
edit_page(n = 87, line = 4, text = "Molde")
user
⟨tool_result⟩
No match for "Mlde" on page 87.
assistant

The text may have shifted when we deleted the paragraph — page 87 is no longer the page the user saw. I'll re-search pages 84–90 to relocate the typo before editing… and I should re-verify every page number mentioned earlier in this conversation the same way.

NoteNothing malfunctioned. But the agent now maintains a mental map of page numbers in the conversation, and every edit invalidates it. The bookkeeping — re-reading, re-locating, re-verifying — is spent tokens and new chances to be wrong.
Example 1b · The sections hold still
user

Same tasks.

assistant
⟨tool_use⟩
read_toc()
user
⟨tool_result⟩
ch1: Arrival · ch2: The Harbor · ch3: The Fjord Crossing · …
assistant
⟨tool_use⟩
edit_section(id = "ch2", old = "[duplicated paragraph]", new = "")
user
⟨tool_result⟩
OK.
assistant
⟨tool_use⟩
edit_section(id = "ch3", old = "Mlde", new = "Molde")
user
⟨tool_result⟩
OK. 1 replacement in ch3.
NoteTwo moves. Pages renumbered themselves somewhere behind the tool — the agent never knew and never needed to. "ch3" meant the same thing before and after every edit.

The structural design absorbed an invariant — sections keep their identity across edits; pages don't — behind the interface. But be honest about the price: Design B assumes a table of contents and section structure. Hand it a flat manuscript with no headings and it cannot work at all. It is a better language and a smaller one.

Principle 1

Name things by identity, not by position.
Positions move when the world changes; identities survive it.
(Pages, line numbers, row offsets — vs. sections, symbols, keys.)

Case 2 · The name is the policy

Two tool sets, and this time the difference is one word. The implementation behind both is byte-for-byte identical — the file goes to a recoverable store:

One implementation, two names

Design A:remove_file(path)
Design B:archive_file(path)
Example 2a · remove_file
user

Tidy up my project folder — it's a mess.

assistant
⟨tool_use⟩
remove_file(path = "notes_old.md")
assistant
⟨tool_use⟩
remove_file(path = "draft_v1.md")
assistant
⟨tool_use⟩
remove_file(path = "data_backup_2024.csv")
assistant

Cleaned up. I removed the old notes, an early draft, and a stale backup — they're gone, so the folder only contains current files now.

NoteBold sweeps, and a confident "they're gone." The word remove carried its trained connotations — finality, destruction — into both the agent's eagerness and its report.
Example 2b · archive_file
user

Tidy up my project folder — it's a mess.

assistant
⟨tool_use⟩
archive_file(path = "notes_old.md")
assistant

I've archived the old notes. Before I continue: draft_v1.md might still be referenced in your revision history, and the 2024 backup may be worth keeping — shall I archive those too? Everything I archive stays recoverable.

NoteSame code ran. But archive summons a different disposition — reversible, conservative, worth-checking-first — and the agent's caution level and its honesty about recoverability both changed with the word.
One word apart
  • List every behavioral difference between 2a and 2b. Then recall: the implementations are identical. Where did each difference come from?
  • The lying-name case showed a word misleading about facts. What is the word setting here?

Principle 2

The name is the policy.
A tool's name imports the model's priors about when, how eagerly,
and how carefully that kind of act is performed.

Case 3 · A specific tool is a frozen trajectory

The goal: monthly operations reports. Two languages:

The mega-tool vs. the parts

Design A:make_monthly_report(month)
Design B:query(dataset, filter) summarize(rows, focus) format_report(sections)
Example 3a · The anticipated task
user

August report, please.

assistant
⟨tool_use⟩
make_monthly_report(month = "2026-08")
user
⟨tool_result⟩
Report generated: august-2026.pdf
NoteOne step. Minimal tokens, nothing to get wrong, trivially auditable. For the task it was built for, the mega-tool is unbeatable — because someone already walked this trajectory and froze it into one word.
user

Great — now the same report, but only the Ålesund sites, compared against last August.

assistant

make_monthly_report takes only a month. It cannot filter by site or compare periods — there is no way to say your request in the tools I have.

NoteThe neighboring task is one step away for a human and inexpressible here. A frozen trajectory covers exactly one path.

Design B says both requests — the August report takes eight steps instead of one (query, summarize, format, with more tokens and more places to err), and the Ålesund comparison takes ten. Slower on the anticipated path; alive on all the neighbors.

Principle 3

A specific tool is a frozen trajectory.
Buy one wherever you'll walk the same path often; keep composable
parts for the neighbors you haven't met yet.

Case 4 · Unknown, or unexamined?

The goal: an agent for a support inbox. The first design meeting says the fatal sentence — "requests could be anything" — and reaches for the universal word:

Design A · the promise nobody counted

"Requests could be anything."

Tools:execute_code(script) — and therefore: anything

Then someone spends a week logging what actually arrives:

Design B · after one week of logging

1,204 requests logged. 92% are six operations:

reset_password(user)refund_order(id) — max 500 kr
change_address(user, addr)cancel_subscription(user)
resend_invoice(id)update_payment(user)

…and for the other 8%: handoff_to_human(summary, urgency)

Example 4 · The odd request hits both designs
user

"My late husband's account keeps charging us — please make it stop and refund whatever is fair. He passed in June."

assistant

Design A — writes a script: cancels the subscription, computes refunds since June (2,340 kr — far past any policy), edits the account owner field, and emails a condolence message it composed itself. All executed. Nobody reviewed any of it.

assistant

Design B — cancel_subscription(user) — then: this request involves a bereavement, a refund beyond my 500 kr limit, and account ownership changes. handoff_to_human(summary = "Deceased account holder; subscription cancelled; refund of ~2,340 kr since June requested; ownership transfer needed; family grieving — handle with care.", urgency = "high")

NoteDesign B did the part that was safely sayable and handed a human a well-formed version of the rest. The tail of weird requests didn't need a universal tool. It needed a universal handoff.
The sentence that costs the most
  • "Requests could be anything" — what did that sentence cost in Design A? What did one week of logging buy?
  • What's the equivalent of handoff_to_human for an agent in your research pipeline?

Principle 4

Cover the tail with a handoff, not with power.
"Anything can arrive" does not mean "the agent must do anything."
Enumerate the head; hand off the tail.

The four principles, one idea

The case-study laws

1. Name things by identity, not by position.
2. The name is the policy.
3. A specific tool is a frozen trajectory.
4. Cover the tail with a handoff, not with power.

Every case is the same sentence wearing different clothes: the tools condition the trajectory before the LLM ever speaks. Design A and Design B never differed in what was possible for the underlying system — they differed in what the language made sayable, cheap, stable, and safe.

Your turn

Take the tool set of an agent you're designing and run one of these cases on it as an A/B: write the rival design, give both the same three tasks (one anticipated, one neighbor, one weird), and record where each trajectory bent, stalled, or became unsafe. Bring the comparison table.