Agentic Research Design Considerations

Begin with a story about the word “better”

We will begin this section as a story. I am sitting in a research presentation. The project has used eye tracking to record where software engineers look while reading code. Those observations have been used to train a specialized model, and the claim is that this smaller model produces better code summaries than frontier models. There are formulas, graphs, objective functions, and quantitative comparisons showing that it scores higher.

My first question is: What does it mean for the summary to be better?

The answer is that it scores higher on the two metrics shown in the graphs. So I ask the next question: What do those metrics actually measure? What they largely measure is how many symbols from the source code appear in the summary.

Now the real problem becomes visible. Why does reproducing more code symbols make a summary better? Better for whom? Better for what task? A summary intended to help a nontechnical reader understand a safety issue may need very different information from one intended to support deployment planning. A summary could score well on symbol density and still be terrible for either purpose.

This sequence of questions is important. We do not begin by accepting the metric and discussing whether the graph is statistically significant. We step backward and ask whether the metric represents the thing the research claims to have improved. A number can be measured precisely while remaining disconnected from the actual task.

The prompt is part of the experiment

My next question in the presentation is: What prompt did you give the frontier model? The answer is that it received only a generic instruction such as summarize.

Now we have another fundamental problem. The specialized model had direct access to the objective through its training, while the frontier model was never told the audience, purpose, or scoring objective. That is not an equivalent comparison.

When I experimented with changing the prompt and stated what the summary was for, the result changed dramatically. A frontier model prompted with the stated objectives performed comparably and appeared capable of outperforming the specialized approach on its own metric. I could also ask it to produce summaries optimized for other arbitrary objectives simply by changing the conversation.

This is the central methodological challenge: prompt selection is part of the experimental design. A result from one prompt does not establish what the model can do across other reasonable prompts. A broad claim such as “produces better summaries” may really mean only: with this prompt, on this task, under this metric and these conditions, we observed a higher score.

We must scope claims to the prompts, data, metrics, and conditions actually tested. We must also acknowledge that an untested prompt or simple adjustment may change the comparison.

Quantification can create false confidence

What counts as good is often task specific and partly qualitative. Simple measures may omit usefulness, clarity, safety, appropriateness for the audience, or the decision the output is supposed to support.

We should not assume that something difficult to measure has been fully captured merely because it has been reduced to a number. When we use a metric, we must explain what it measures, what it does not measure, and why it is connected to the actual task. Proxy metrics should be paired with qualitative or task-based evidence when they cannot establish usefulness by themselves.

Some research questions require human-centered evaluation

If the goal is to produce summaries for people who are not programmers, then those people and their needs belong in the evaluation. Can they understand the summary? Does it support the decision they need to make? Does it omit important information or create unwarranted confidence?

Computer scientists may prefer clean automated scores, but the research question may demand human response. LLM-as-judge approaches can contribute evidence, but they introduce their own prompts, biases, assumptions, and validation requirements. They do not make the qualitative question disappear.

Formalism does not rescue an invalid question

A paper can contain extensive mathematics, graphs, objective functions, and polished statistical results while still measuring the wrong construct or comparing against an unfair baseline. Formal presentation does not repair an invalid definition of quality.

This is where researchers must be willing to step back. In the original story, the paper had already been accepted and contained many of the formal artifacts people expect to see. Yet the foundational questions—what “better” means, whether the metric represents the task, and whether the baseline received a fair prompt—remained unresolved.

Before trusting the apparent rigor of a result, we have to ask whether the task, prompt, metric, and claim actually fit together. Otherwise, a technically impressive study may be wrong in the narrow sense and misleading in the broader sense: it can give later researchers confidence in a conclusion the experiment did not actually establish and direct work down the wrong path.

Probabilistic systems require probabilistic evidence

LLMs are probabilistic. Success in 10 or 100 trials does not guarantee success in trial 101. Agent trajectories can diverge because one response changes the next action, which changes the next observation, which changes the conversation that follows.

Evaluation should therefore include repeated trials, multiple prompt formulations, diverse inputs and operating conditions, uncertainty estimates, and attention to rare failures rather than only average performance.

Safety raises the standard further. Not observing a harmful outcome is not proof that the system is safe. Low-frequency failures may be missed by ordinary testing, especially when agents have powerful tools. We must examine tail risks, adversarial conditions, tool boundaries, containment assumptions, and the consequences of failure.

The research lesson is the same as the technical lesson: we have to reason carefully about the conversation. Which prompt was tested? Which information was present? Which tools were available? What sequence of responses occurred? And how much evidence do we really have that the same behavior will persist when the conversation changes?