Build It and Try It: Exploring Agent Designs for Your Domain

75 min

Overview

Yesterday you committed to an agent concept and a question about it. Today you build the agent and use it — not a demo of the Forge, your actual agent, on a real or realistically simulated task from your domain. Then you build it differently: a different toolset, a swapped tool, the training document in or out — quick builds, quick trials, rough tallies. This morning taught you that tools are the language an agent thinks in; this afternoon you find out, by trying, which parts of that language matter for your agent. The output is not a polished system. It is a stack of build-notebook entries and an emerging hunch: "the key seems to be ⟨this kind of tool / design / approach⟩ — look what happens when we add it."

The spirit of the thing

This is a sketchbook, not a lab. Build fast, try it yourself, write one honest line, build again. "It worked in 6 turns" and "it flailed until we added the lookup tool" are exactly the right resolution of evidence for this week.

Rough tallies to keep per trial: did it work? how many turns? how many tokens? what did it do that surprised you?

The exercise at a glance

1
Sketch the action language your agent needs: 3–5 tools, plus one training document.
2
Build it in the Forge and USE it — run a real task, read the whole trajectory.
3
Build it differently: change the toolset, a tool, or the training doc — try the same task again.
4
If time allows, a third build — chase whatever the first two made you curious about.
5
Read your notebook lines and name the hunch: what seems to matter?

Part 1 · Sketch the language — fifteen minutes, not more

For your agent concept, sketch the smallest toolset that could possibly do the job: three to five tools. Write each as if the agent will read it — because it will.

Each tool, quickly but deliberately

namethe verb the agent will think in — "search_cases" and "find_precedent" pull the conversation in different directions

parameterswhat the agent must decide before it can act

result formatwhat comes back, and in what shape — the result is next turn's context

boundarywhat the tool refuses to do — the capability you leave out is a design decision too

Then write one training document — a page at most: the domain procedure or judgment your agent most needs and a frontier model least likely knows. Brief it like a brilliant new employee: context, procedure, the two mistakes everyone makes.

Don't perfect any of this. You are going to change it within the hour — that's the exercise.

If you want a thinking partner for the sketch, open the Design Coach: /exercises/design-coach — a chat coach that works from the course's agent-design guide and will walk through your tool names, parameters, result formats, and training document with you. Use it for a few minutes of sharpening, not the whole fifteen — the sketch only has to be good enough to build.

Scan to open the Design Coach — ai-agents-seminar.vercel.app/exercises/design-coach

Part 2 · Build it and use it

Open the Agent Forge: /exercises/agent-forge — or scan the QR code on the next card. Give the agent one real task from your domain — or a realistically simulated one: fake data in the tools is fine, and a tool you play by hand (you type what the system would return) is fine too.

Scan to open the Agent Forge — ai-agents-seminar.vercel.app/exercises/agent-forge

The first trial

1
Declare your tools; let the Forge write the code.
2
Run the task end to end. Do not steer, do not rescue.
3
Use it like a user, not a builder: is this output you would actually accept? Would you trust it enough to act on?
4
Notebook line: what you built, whether it worked, turns, tokens, and the one moment worth remembering.

Part 3 · Build it differently

Now change something real — not a tweak, a different design:

Ways to build it differently — pick what you're curious about

a different toolsetcoarser or finer tools, or a different decomposition of the work entirely

one tool swappedreplace the tool the agent leaned on hardest with a different shape of the same capability

the training documentin versus out — is the domain briefing earning its context, or does the model already know?

the result formatraw dump versus digest — feed the agent a different future prompt and watch what changes

Same task, new build, another honest notebook line. If something surprises you, chase it — a third quick build in the direction of the surprise beats a disciplined march through a plan. This is exploration; follow what pulls.

Part 4 · Name the hunch

Read your notebook lines side by side and answer, in one or two sentences: what seems to matter? Which tool, design choice, or approach made the difference between the build that worked and the build that flailed — and does that line up with the question you committed to yesterday?

The shape of a good hunch

too vague"better tools help"

right"with the domain briefing in, it solved the task in 6 turns; without it, it invented a workflow that doesn't exist in our field — the key seems to be teaching the procedure, not adding more tools"

That sentence — hunch plus the trials behind it — is the seed of your paper's Approach section, and Thursday's proof-of-concept session will test whether it holds up.

Before we regroup

💾 Capture

Bring your notebook.

Write down:

1The builds: what you tried, one line each — worked or not, turns, tokens, the memorable moment.
2Two verbatim excerpts: the best moment of your best build, and the most instructive failure of any build.
3The hunch — one or two sentences on what seems to matter.

Be ready to discuss:

Later today you design the study you WOULD run to test this hunch properly. Keep the rough tallies you used — turns, tokens, worked-or-not — you'll ask this afternoon whether they could grow into real measures.