Build your own agent harness, part 4: one while turns a chat into an agent
Last time, we left things broken. On purpose, I promise 😇 Our model learned to ask for a tool, our harness ran it and sent the result back. One round trip. And when the model wanted a second one, we simply stopped listening: the answer was null.
The code comment even said it: "Step 4 turns this if into a while". Well, here we are. And this tiny change is, I think, the most important one of the whole series. It's the difference between a chat that can use a tool and an agent that works on its own.
if becomes while
Here is the heart of Agent.java now:
// The agentic loop: as long as the model asks for tools, we run them and call it again.
while (answer.hasToolExecutionRequests()) {
for (var call : answer.toolExecutionRequests()) {
display.toolCall(call);
String result = execute(call);
display.toolResult(result);
history.add(ToolExecutionResultMessage.from(call, result));
}
answer = send(model, history);
}
Look closely: it's the code of article 03, with one word changed. As long as the model asks for tools, we run them, we add the results to the history, and we call the model again. When the answer contains no tool request, only text, the turn is over.
And who decides when it's over? Not our code. The model. It can take one step or ten, our harness just follows. That's it, that's an agent: a loop. No magic, remember? 😉
Tools become data
With a loop, a clock alone is a bit sad. So this step swaps it for two tools that make sense for a coding agent: list_files and read_file. And the system prompt now describes a coding assistant:
final String SYSTEM_PROMPT = """
You are a coding assistant working in the current directory.
Use the tools to look at the files before answering. Be concise.
""";
With several tools, a switch in execute() would quickly get messy. So tools become data: a small registry.
// A tool = what we tell the model (spec) + what we actually run.
interface ToolFunction {
String run(String argument) throws Exception;
}
record Tool(ToolSpecification spec, ToolFunction function) {}
final ObjectMapper json = new ObjectMapper();
// The tool registry. Step 5 will show that one generic tool can replace all of them.
final Map<String, Tool> TOOLS = Map.of(
"list_files", tool("list_files", "Lists the entries of a directory", "path", this::listFiles),
"read_file", tool("read_file", "Reads a text file", "path", this::readFile));
A tool is two things: a specification, what the model sees (we met it in article 03), and a function, what we actually run. The map links them by name. Adding a tool is one line. (And yes, there's a teaser in that comment. Patience 😜)
Unlike our clock, these tools need a parameter: which directory, which file. The tool() helper builds both halves:
// Builds a tool that takes a single string parameter.
Tool tool(String name, String description, String parameter, ToolFunction function) {
var spec = ToolSpecification.builder()
.name(name).description(description)
.parameters(JsonObjectSchema.builder().addStringProperty(parameter).required(parameter).build())
.build();
return new Tool(spec, arguments -> function.run(json.readTree(arguments).path(parameter).asText()));
}
The specification now declares a parameter: path, a required string. Remember, the model sends its arguments as a JSON string (we saw "arguments": "{}" last time). The helper parses it with Jackson and hands the plain value to our Java method.
And the tools themselves? Plain Java, nothing agent-specific:
String listFiles(String path) throws IOException {
try (var entries = Files.list(Path.of(path))) {
return entries.map(p -> Files.isDirectory(p) ? p.getFileName() + "/" : p.getFileName().toString())
.sorted().collect(Collectors.joining("\n"));
}
}
String readFile(String path) throws IOException {
return Files.readString(Path.of(path));
}
Finally, send() attaches every specification of the registry to each request:
var request = ChatRequest.builder().messages(history).toolSpecifications(TOOLS.values().stream().map(Tool::spec).toList()).build();
Errors are results, not exceptions
Here is the new execute():
// The harness, not the model, runs the code.
String execute(ToolExecutionRequest call) {
Tool tool = TOOLS.get(call.name());
if (tool == null) return "Error: unknown tool " + call.name();
try {
return tool.function().run(call.arguments());
} catch (Exception e) {
return "Error: " + e; // the model sees the error and can try something else
}
}
It never throws. A missing file, an unknown tool: the harness turns the problem into an Error: … string and sends it back like any other result. Why? Because in a loop, an exception would kill the whole turn. A string, on the other hand, the model can read. And then it can adapt: list the directory, try another name, ask you. A failing tool must never stop the loop.
Keeping the reasoning through the loop
One last line, on the model builder, and it deserves a bit of care:
.include(List.of("reasoning.encrypted_content")) // ...including the model's reasoning between loop turns
gpt-6-luna that I use in my tests, as a lot of models nowadays, is a reasoning model: it thinks before answering, and in a loop, it thinks between tool calls too. "I listed the directory, there's a README, let's read it, then the pom…". That thinking is part of what makes the next step smart.
But remember article 01: with store(false), the server keeps nothing. If the reasoning doesn't come back to us, it's gone, and at each turn of the loop the model would start again without its previous thoughts. This line asks the API to return the reasoning in encrypted form, so our harness can send it back with the rest of the history. We can't read it, the provider can.
You may remember a mysterious reasoning item in the JSON of article 01. That's the one. In a simple chat it didn't matter much; in a loop, it does. So now we ask for it explicitly. Same statelessness story as article 01, one level deeper.
Run it
This time, run the agent from the directory you want it to explore: it reads files relative to where it runs.
export LLM_API_KEY=...
jbang 04-agentic-loop/Agent.java --inspect
Try it
Ask:
Explore this project and summarize it.
And watch. Several ● list_files and ● read_file in a row, each one chosen by the model, then a summary. You didn't say which files to read, nor how many. The model decided. Look at the grey line too: one question, several calls.
Now let's be mean again:
Read the file NOTES.md.
(a file that doesn't exist, of course). You get └ Error: java.nio.file.NoSuchFileException: NOTES.md… and the loop goes on. The model reads that error like any result and reacts: it lists the directory, picks a close match, or asks you. No crash, no stack trace on your screen.
And the cliffhanger of article 03? Our clock is gone, but the same idea works:
Read the README, then read every file it mentions.
Several read_file calls in a row, decided by the model. No more null answer: the loop goes on as long as the model asks.
What the JSON shows
In inspect.log, every request after a tool call now ends with a reasoning item carrying encrypted_content, then the function_call item(s) and their function_call_output item(s). The model's thoughts, its requests, our answers: all in the history, all sent again.
Loops multiply the bill
Remember article 02? Each call resends the whole history. Now one question can mean five, ten calls. And the history doesn't only hold our chat anymore: it holds tool results. Entire file contents. Every later call pays for them again.
Keep an eye on the grey line during "Explore this project": the input tokens per call climb fast. An agent that reads a lot is an agent that costs a lot 💸
Limits
- Nothing bounds the loop. If the model keeps asking for tools, the harness keeps running them, and paying for them.
- Reading a big file puts all of it in the context, and every later call pays for it again.
read_fileandlist_filesaccept any path the process can read, anywhere on the disk. Harmless here, since they only read. It becomes a real question as soon as tools can change things.- The
tool()helper only supports tools with a single string parameter. Enough for now.
As usual, the code, a README to run it and things to try are in the step-04 folder.
Next time
Two tools, two Java methods. Every new capability means new code in the harness… unless a single tool could do it all. What if we just gave the model a shell?
Tell me what you think 💞
