Module 2 · Tools, Knowledge, Memory & Research Design · scripted
Tool Design Case Studies
Four cases, one method
Everything so far — names as pointers, descriptions as paid definitions, the tool set as a language — becomes concrete the same way: take one goal, design two tool sets for it, and watch what happens to the trajectory. Four cases. Each ends in a principle you can reuse.
Case 1 · Editing a book
The goal: revise a 300-page manuscript. Two languages for it:
Two tool sets for one manuscript
Delete the duplicated paragraph in chapter 2, then fix the typo "Mlde" in the chapter about the fjord crossing — I saw it around page 87.
⟨tool_use⟩
read_page(n = 41)⟨tool_result⟩
[page 41 text: …the duplicated paragraph appears here…]⟨tool_use⟩
delete_from_page(n = 41, line = 12)⟨tool_result⟩
OK. Document is now 299 pages.⟨tool_use⟩
edit_page(n = 87, line = 4, text = "Molde")⟨tool_result⟩
No match for "Mlde" on page 87.The text may have shifted when we deleted the paragraph — page 87 is no longer the page the user saw. I'll re-search pages 84–90 to relocate the typo before editing… and I should re-verify every page number mentioned earlier in this conversation the same way.
Same tasks.
⟨tool_use⟩
read_toc()⟨tool_result⟩
ch1: Arrival · ch2: The Harbor · ch3: The Fjord Crossing · …⟨tool_use⟩
edit_section(id = "ch2", old = "[duplicated paragraph]", new = "")⟨tool_result⟩
OK.⟨tool_use⟩
edit_section(id = "ch3", old = "Mlde", new = "Molde")⟨tool_result⟩
OK. 1 replacement in ch3.The structural design absorbed an invariant — sections keep their identity across edits; pages don't — behind the interface. But be honest about the price: Design B assumes a table of contents and section structure. Hand it a flat manuscript with no headings and it cannot work at all. It is a better language and a smaller one.
Principle 1
Name things by identity, not by position. Positions move when the world changes; identities survive it. (Pages, line numbers, row offsets — vs. sections, symbols, keys.)
Case 2 · The name is the policy
Two tool sets, and this time the difference is one word. The implementation behind both is byte-for-byte identical — the file goes to a recoverable store:
One implementation, two names
Tidy up my project folder — it's a mess.
⟨tool_use⟩
remove_file(path = "notes_old.md")⟨tool_use⟩
remove_file(path = "draft_v1.md")⟨tool_use⟩
remove_file(path = "data_backup_2024.csv")Cleaned up. I removed the old notes, an early draft, and a stale backup — they're gone, so the folder only contains current files now.
Tidy up my project folder — it's a mess.
⟨tool_use⟩
archive_file(path = "notes_old.md")I've archived the old notes. Before I continue: draft_v1.md might still be referenced in your revision history, and the 2024 backup may be worth keeping — shall I archive those too? Everything I archive stays recoverable.
- List every behavioral difference between 2a and 2b. Then recall: the implementations are identical. Where did each difference come from?
- The lying-name case showed a word misleading about facts. What is the word setting here?
Principle 2
The name is the policy. A tool's name imports the model's priors about when, how eagerly, and how carefully that kind of act is performed.
Case 3 · A specific tool is a frozen trajectory
The goal: monthly operations reports. Two languages:
The mega-tool vs. the parts
August report, please.
⟨tool_use⟩
make_monthly_report(month = "2026-08")⟨tool_result⟩
Report generated: august-2026.pdfGreat — now the same report, but only the Ålesund sites, compared against last August.
make_monthly_report takes only a month. It cannot filter by site or compare periods — there is no way to say your request in the tools I have.
Design B says both requests — the August report takes eight steps instead of one (query, summarize, format, with more tokens and more places to err), and the Ålesund comparison takes ten. Slower on the anticipated path; alive on all the neighbors.
Principle 3
A specific tool is a frozen trajectory. Buy one wherever you'll walk the same path often; keep composable parts for the neighbors you haven't met yet.
Case 4 · Unknown, or unexamined?
The goal: an agent for a support inbox. The first design meeting says the fatal sentence — "requests could be anything" — and reaches for the universal word:
Design A · the promise nobody counted
"Requests could be anything."
Then someone spends a week logging what actually arrives:
Design B · after one week of logging
1,204 requests logged. 92% are six operations:
…and for the other 8%: handoff_to_human(summary, urgency)
"My late husband's account keeps charging us — please make it stop and refund whatever is fair. He passed in June."
Design A — writes a script: cancels the subscription, computes refunds since June (2,340 kr — far past any policy), edits the account owner field, and emails a condolence message it composed itself. All executed. Nobody reviewed any of it.
Design B — cancel_subscription(user) — then: this request involves a bereavement, a refund beyond my 500 kr limit, and account ownership changes. handoff_to_human(summary = "Deceased account holder; subscription cancelled; refund of ~2,340 kr since June requested; ownership transfer needed; family grieving — handle with care.", urgency = "high")
- "Requests could be anything" — what did that sentence cost in Design A? What did one week of logging buy?
- What's the equivalent of handoff_to_human for an agent in your research pipeline?
Principle 4
Cover the tail with a handoff, not with power. "Anything can arrive" does not mean "the agent must do anything." Enumerate the head; hand off the tail.
The four principles, one idea
The case-study laws
1. Name things by identity, not by position. 2. The name is the policy. 3. A specific tool is a frozen trajectory. 4. Cover the tail with a handoff, not with power.
Every case is the same sentence wearing different clothes: the tools condition the trajectory before the LLM ever speaks. Design A and Design B never differed in what was possible for the underlying system — they differed in what the language made sayable, cheap, stable, and safe.
Take the tool set of an agent you're designing and run one of these cases on it as an A/B: write the rival design, give both the same three tasks (one anticipated, one neighbor, one weird), and record where each trajectory bent, stalled, or became unsafe. Bring the comparison table.