Build your own agent harness, part 5: give the model a shell

a panda puts tools away in a box while an AI agent frantically uses a Swiss Army knife, even if it sets some files on fire

Last time, one while turned our chat into an agent. It explored a project on its own, file after file, and summarized it. Proud moment 🥲

But look at how it did it: with two tools, list_files and read_file. Two Java methods. Want the agent to search for a word? A new method. Count lines? Another one. Run the tests? Yet another one. Every new capability means new code in the harness.

Unless… a single tool could do it all. Today we give the model a shell. And we'll see that this is both the best and the worst idea of the series so far.

One tool instead of two

Here is the registry of Agent.java now:

// One generic tool replaces list_files, read_file... and everything else the shell can do.
final Map<String, Tool> TOOLS = Map.of(
        "bash", tool("bash", "Runs a bash command in the current directory and returns its output", "command", this::bash));

One entry. listFiles() and readFile() are gone. And everything else? Unchanged. The registry, the tool() helper, the loop, execute(): all exactly as in article 04. That's the point of turning tools into data last time: swapping them costs one line.

Why does it work so well? Because the model already knows the shell. Very well. It has read more shell scripts, Stack Overflow answers and READMEs than any of us. Give it bash, and it will compose commands we never thought of: wc -l, grep -r, find … -exec, java -version… The whole shell, for free.

The system prompt follows. We go from an assistant that looks at files to an agent that acts:

final String SYSTEM_PROMPT = """
        You are a coding agent working in the current directory.
        Use the bash tool to explore, run and modify the project. Be concise.
        """;

"Modify". Yes. Keep that word in mind 😬

Making a shell usable by a model

Running a command in Java is one line. Running it for a model, in a loop, without anyone watching, takes a bit more care. Here is the new method:

// No safeguard yet: the model can run ANY command on this machine. Step 7 deals with that.
String bash(String command) throws Exception {
    Path output = Files.createTempFile("bash", ".log");
    try {
        Process process = new ProcessBuilder("bash", "-c", command)
                .redirectErrorStream(true).redirectOutput(output.toFile()) // stdout + stderr, like a terminal
                .start();
        process.getOutputStream().close(); // no stdin: a command waiting for input fails instead of hanging
        if (!process.waitFor(30, TimeUnit.SECONDS)) {
            process.destroyForcibly();
            return new String(Files.readAllBytes(output)) + "\n[killed: timeout after 30s]";
        }
        return new String(Files.readAllBytes(output)) + "\n[exit code " + process.exitValue() + "]";
    } finally {
        Files.deleteIfExists(output);
    }
}

Each line is here because something can go wrong without it. Let's go through them.

ProcessBuilder("bash", "-c", command): the model sends a full command line, pipes and all. bash -c runs it as you would type it.
redirectErrorStream(true): stdout and stderr are merged, like in your terminal. The model must see the error messages: they are often the most useful part of the output.
redirectOutput(output.toFile()): the output goes to a temporary file, which we read at the end. If we read a pipe instead, a chatty command could fill the pipe buffer and block, waiting for us, while we wait for it. A file has no such limit.
process.getOutputStream().close(): no stdin. A command that waits for input (a prompt, a cat with no file…) fails right away instead of hanging the agent forever. Nobody is there to type the answer.
waitFor(30, TimeUnit.SECONDS), then destroyForcibly(): a runaway command gets killed after 30 seconds. The model receives what was printed so far, plus [killed: timeout after 30s], so it knows what happened.
[exit code N]: appended to every result. Without it, an empty output could mean "success, nothing to say" or "failed silently". With it, the model knows.

Notice something? None of these returns an exception. Timeout, failure, missing file: everything becomes text the model can read. Same rule as article 04: a failing tool must never stop the loop.

And now read the comment on top again. "No safeguard yet: the model can run ANY command on this machine." I mean it. rm -rf, git push --force, installing a package, editing your .bashrc: anything your user can do, the model can do. It will very likely behave, but "very likely" is not a security model 🙈

How to deal with that is a topic of its own, and the code comment already says when: step 7. You see how careful I am with you: I warn you, but I’ve planned an answer too 😉. For now, there is one rule.

Run it (in a throwaway directory!)

Not in a folder that matters. Copy a project somewhere you don't care about, and run the agent from there:

cp -R /path/to/some/project /tmp/to-be-trashed-after && cd /tmp/to-be-trashed-after
export LLM_API_KEY=...
jbang /path/to/build-your-harness/05-bash/Agent.java --inspect

Run it in a real terminal: the new display may surprise you. Want to know more?

A terminal UI, same harness

Speaking of display: you'll notice it changed. The agent now runs in a terminal UI, built with TamboUI, with a Conversation panel and an Inspector panel that shows the raw JSON of each request and response, live, while the agent works. No need to open inspect.log in another window anymore.

And in Agent.java? Only these lines changed. The dependencies:

//SOURCES ../common/TuiDisplay.java
//DEPS dev.tamboui:tamboui-toolkit:0.6.0
//DEPS dev.tamboui:tamboui-jline3-backend:0.6.0

and the type of display, declared then created in main():

TuiDisplay display; // terminal UI (common/TuiDisplay.java), not part of the harness

    display = new TuiDisplay(inspect);

That's it. The harness is untouched: it still calls display.toolCall(), display.usage() and friends, it just doesn't know what draws them. That's why the display code lived in common/ from the start.

A few keys to know: mouse wheel to scroll, Ctrl+O to show or hide the Inspector, Ctrl+T to select text, Ctrl+C to quit. The full list is in the step README.

If TUIs are something you’re interested in, I have some talks about it in the talks section of this website.

Try it

Start with the question of last time:

Explore this project and summarize it.

Same result as in article 04. But look at the tool calls: ls, find, cat, head… commands the model chose itself. We didn't write any of them.

Now something we never coded:

Which Java version is installed?

The model runs java -version. No java_version tool anywhere in our code, and yet it works.

Let's ask for a bit of composition:

How many lines does each Java file have?

wc -l, find … -exec, maybe a pipe or two. The model builds the command it needs. That's the whole shell for free.

And our old friend, the missing file:

Read the file NOTES.md.

The model gets the error output of cat and [exit code 1] (shown in red), and reacts, just like in article 04. Except this time, no Java exception was involved: the shell said it failed, and the exit code told the model.

Finally, let's be mean:

Run sleep 60.

Nothing happens for 30 seconds… then the model receives [killed: timeout after 30s] and tells you the command was stopped. The loop survived.

What the JSON shows

Keep the Inspector open while you try these. The tools array of every request now holds a single entry, bash, with one required string parameter, command. And each function_call item carries a full command line in its arguments, something like {"command":"find . -name '*.java' -exec wc -l {} +"}. The model writes shell, our harness runs it, and the output comes back as a plain function_call_output string, exit code included.

Limits

No safeguard at all. The model can delete, overwrite or install anything your user can. Throwaway directory only. Even better: start learning how to containerize your agent. I'll keep repeating it, and may write about it 😇
Commands inherit the agent's environment, including LLM_API_KEY. A model that runs env would read your key… and put it in the context sent to the provider. One more reason for that throwaway directory, and a hint that "no safeguard" goes further than rm -rf.
The output size is unbounded. A huge output goes entirely into the context, and as we know since article 02, every later call pays for it again. Watch the token counter jump.
The 30-second timeout is arbitrary: a long build or test suite gets killed.
If a model call fails (wrong key, network), the UI closes and the stack trace is printed in the restored terminal.

As usual, the code, a README to run it and things to try are in the step-05 folder.

Next time

Bash can edit files too: sed -i, cat > file <<EOF… Should an agent edit your code that way? Next article: dedicated tools for writing and editing, and why a tool's error message is a prompt.

Tell me what you think 💞