Module 2 · Building Agents · scripted

The Conversation with the Computer

20 minoutcomes: agent-architectures, tool-design
🎤 Talk view← Cockpit

Most agents talk to computers

The agents we have built so far steer a human. But most AI agents we actually think about have a special property: the LLM spends most of its time having a conversation with a computer, not with a human.

The discussion it has with the computer is the same shape as the one it had with the cook: it asks the computer to take an action; the computer takes the action; the computer replies with an updated state of the world. The way the LLM asks is by using a tool — a tool call is a message to the computer that causes an action to take place. The result of the tool call comes back as a description of what changed.

The participants

A human says "go"

A conversation like this is often kicked off by a person stating a goal — and then stepping out of the loop:

Example 1 · Find my trip updates
user

You have the following tools available for use:

  1. search_email( query )
  2. read_email( id )

Task: Look through my email and find any updates related to my Molde, Norway trip.

assistant
⟨tool_use⟩
search_email(query="Molde")
user
⟨tool_result⟩
3 matches: [12] "Hotel Alexandra – booking confirmed",
[47] "SAS – schedule change for your flight",
[51] "Fwd: fjord kayaking?"
NoteThat tool call was the LLM's action — a message asking the computer to do something. The result is the computer's report: a description of what it found in the world.
assistant
⟨tool_use⟩
read_email(id=47)
user
⟨tool_result⟩
"Your SAS flight WF562 Oslo–Molde on 24 Aug now departs 14:05
(previously 11:20). Your connection from Amsterdam is unaffected."
NoteLook at which email it opened first. The schedule change is the one that can break a trip — the LLM chose its next action based on what the previous result taught it about the world.
assistant

One update matters: your Oslo–Molde flight on 24 Aug now departs at 14:05 instead of 11:20 — you'll land about three hours later. The hotel booking is confirmed, and the kayaking thread has no changes. You may want to warn whoever is meeting you.

NoteThe final turn is the whole point of the trajectory: the conversation has reached a state that says the goal is achieved — updates found, assessed, and summarized.

Nobody says "go"

The kickoff doesn't need a human at all. The first message of the conversation can be composed automatically — an email arriving with a certain subject line, a sensor crossing a threshold, a timer firing:

Example 2 · The trigger at 06:12
user

[Automated trigger] New email received matching rule "airline schedule change". New email: id=88.

You have the following tools available for use:

  1. read_email( id )
  2. get_calendar( date )
  3. update_event( id, changes )
  4. send_message( to, text )

Task: Keep the calendar consistent with all bookings.

assistant
⟨tool_use⟩
read_email(id=88)
user
⟨tool_result⟩
"SAS: flight WF562 Oslo–Molde on 24 Aug now departs 14:05, arriving
15:10."
assistant
⟨tool_use⟩
get_calendar(date="2026-08-24")
user
⟨tool_result⟩
[e3] 11:20–12:25 "Flight WF562 OSL→MOL"; [e4] 13:00 "Pickup at Årø
airport — Bjørn"
NoteThe calendar reveals a second problem the email never mentioned: Bjørn is picking someone up at 13:00 for a flight that now lands at 15:10.
assistant
⟨tool_use⟩
update_event(id="e3", changes="14:05–15:10")
assistant
⟨tool_use⟩
send_message(to="Bjørn", text="Flight WF562 on 24 Aug now lands 15:10
— pickup moved to 15:15 at Årø. Sorry for the change!")
assistant

Calendar updated and Bjørn notified. All bookings and calendar entries are consistent again.

NoteNo human typed anything — at 06:12 an arriving email started this conversation. Every action after that was the LLM steering toward one state: "the calendar is consistent with the bookings."
Compare the two kickoffs
  • In Example 1 a human stated the goal; in Example 2 a rule did. What else differs between the two conversations? What is identical?
  • In Example 2, where did the agent learn about the pickup problem — the email, or its own tool use?

Still a flipped interaction

Step back and recognize the pattern. This is the cooking conversation with one participant swapped out. The LLM leads; it takes one action at a time; each action elicits new information about the world; and the answer to each action conditions the next one. The only change: the participant performing the actions and reporting back is now the computer itself, with no human intermediary carrying messages between them.

The same pattern, two participants

Cooking (human in the loop)Email agent (computer in the loop)
───────────────────────────────────────────────────────────
"Whisk the eggs, tell mesearch_email(query="Molde")
what you see"
"Done — pale and foamy"3 matches: [12], [47], [51]
adapts: next step uses theadapts: reads the schedule-change
batter's stateemail first
goal state: sveler on the plategoal state: updates found and summarized

The human's role has moved: from performing every action to stating the goal — or writing the rule that states it automatically. What remains constant is the machinery underneath: a conversation, a trajectory, and an LLM writing the messages that steer it.

Your turn

Sketch, in one page, a computer-in-the-loop agent for your research domain: what event or person kicks off the conversation, three to five tools with parameters (name · description · parameters, as precise as the GRILLVENN), and — hardest — what conversation state means "done."