Build your own agent harness, part 6: the harness is the security boundary

Last time, we gave the model a shell. One bash tool, and our agent could explore, count, compile, run anything. With a comment on top that said it all: "No safeguard yet: the model can run ANY command on this machine."

I promised an answer. Here it is 😇

And let's start with what does not work: asking nicely. You could write in the system prompt "never delete anything", "always be careful"… The model will very likely comply. But a system prompt is a wish, not a rule. The model is not a security boundary. The harness is: the code that decides what actually runs.

Today we add three guardrails around our bash tool. Same single tool as article 05, three new checks around it.

A word of warning, though: the goal here is to understand the concept of guardrails, not to ship a complete implementation. A real one goes much further than what fits in a discovery article like this one.

The guardrail settings

Here they are, at the top of Agent.java:

// Guardrails: the harness, not the model, is the security boundary.
final int MAX_TOOL_ROUNDS = 25; // per user request: a runaway loop costs time and money
// Read-only programs run without asking. Any other command needs the user's approval.
final Set<String> READ_ONLY = Set.of("ls", "cat", "head", "tail", "wc", "grep", "pwd", "tree");
final Pattern HIDDEN_COMMANDS = Pattern.compile("[`$<>\\n]"); // substitutions, redirections, extra lines

A limit, a list, a pattern. Let's see what each one is for.

Guardrail 1: read-only commands run freely

The obvious safe answer would be: ask the user before every command. Try it for five minutes. The agent explores a project with twenty ls and cat, and you press y twenty times. By the twenty-first, you stop reading what you approve. Or you switch the whole thing off. A guardrail nobody can live with is no guardrail at all.

So a short list of read-only programs runs without asking: ls, cat, head, tail, wc, grep, pwd, tree, plus git status, git diff and git log. Everything else asks.

Simple, right? Check that the command starts with ls, and…

ls; rm -rf ~

…it starts with ls 🙃

Parsing a shell command is harder than it looks. Here is how our harness decides:

// Auto-approved only if it can't change anything: nothing hidden ($(...), redirections...), and every
// chained or piped command is read-only. "ls && cat README.md | head" passes, "ls; rm -rf ~" does not.
boolean isReadOnly(ToolExecutionRequest call) throws Exception {
    String command = json.readTree(call.arguments()).path("command").asText();
    if (HIDDEN_COMMANDS.matcher(command).find()) return false;
    for (String segment : command.split("&&|\\|\\||[;|&]")) {
        String part = segment.strip();
        String program = part.split("\\s+")[0];
        if (!READ_ONLY.contains(program) && !part.matches("git (status|diff|log)( .*)?")) return false;
    }
    return true;
}

Three steps:

• Refuse anything that can hide another command. That's the HIDDEN_COMMANDS pattern: ` and $ for command substitution (cat `rm -rf ~` or cat $(…)), < and > for redirections (cat secrets > /tmp/x writes a file), and line breaks, because a second line is a second command. If any of these appears, the command is not read-only. Period.
• Split chained commands: &&, ||, ;, & and |. Each one separates two programs.
• Require every part to be read-only. ls -la && cat README.md | head passes. ls; rm -rf ~ doesn't. cat a | sh doesn't either: sh isn't in the list.

Is this split perfect? No. A ; inside quotes (grep ";" file) will be cut in the wrong place. But look at what happens then: the "program" of the second part is " followed by something, not in the list, so the command asks. When the parsing is wrong, it falls back on the safe side. That's the property that matters.

A small story about that for loop. My first version accepted no chaining at all: any &&, | or ;, and the command asked. Safe, very safe. Except models chain commands all the time. ls -la && cat README.md, grep -r TODO . | head… Almost every command asked for approval, and I was back to pressing y without reading. Too strict a guardrail gets switched off. Splitting and checking each part kept the safety and made the agent usable again.

Guardrail 2: a human approves everything else

Now the gate itself, at the top of execute():

if (!isReadOnly(call) && !display.confirm(call)) {
    return "Denied: the user refused this action.";
}

If the command isn't read-only, the harness asks you. display.confirm(call) is display code (it lives in common/TuiDisplay.java, like the rest of the UI): it shows the full command with Allow? [y]es / [n]o, and blocks the agent until you answer. The important part is when: before it runs. You see the exact command, not a summary written by the model.

And if you say no? No exception, no crash. The refusal goes back to the model as the tool result, a plain string: "Denied: the user refused this action." Same rule as in article 04: everything is a result the model can read. And the model adapts. It asks you what to do, tries something else, or explains why it needed that command.

Guardrail 3: a limit on the loop

The last one is about the while of article 04. Nothing bounded it: a model stuck in a loop could keep calling tools, and as we know since article 02, every call costs more than the previous one 💸

// The agentic loop, now bounded: at most MAX_TOOL_ROUNDS rounds of tool calls per request.
int rounds = 0;
while (answer.hasToolExecutionRequests() && rounds++ < MAX_TOOL_ROUNDS) {
    // ... unchanged: run the tools, send the results ...
}
if (answer.hasToolExecutionRequests()) {
    // Limit reached: every requested call still needs a result, or the next request is invalid.
    for (var call : answer.toolExecutionRequests()) {
        history.add(ToolExecutionResultMessage.from(call, "Not executed: tool round limit reached."));
    }
    display.answer("Stopped: more than " + MAX_TOOL_ROUNDS + " tool rounds for one request.");
} else {
    display.answer(answer.text());
}

25 rounds of tool calls per user request, then we stop. Counting is easy. The subtle part is after the loop.

When we stop, the last answer of the model still contains tool requests. We won't run them, but they are already in the history. And remember article 03: every function_call must have its function_call_output, linked by its call_id. If we leave a call without an output, the next request, when you type your next message, is invalid, and the API rejects it. So each pending call gets a result anyway: "Not executed: tool round limit reached." The model will see it next turn, and the history stays consistent.

Stopping a loop cleanly means leaving the history in a state the API accepts. That's the kind of detail you only learn by writing the harness yourself.

Instructions are not guardrails

Finally, two new lines in the system prompt:

        Some actions need the user's approval: if one is denied, do not retry it, ask the user instead.
        Prefer one simple command per bash call, without chaining, pipes or redirections.

Useful: simple commands are easier for you to approve, and a model that retries a refused command over and over is annoying. But notice: these are instructions. The model may follow them or not. The allowlist must stay safe even if the model ignores every word of this prompt. That's the whole difference between the two.

And the comment on top of bash() changes too. "No safeguard yet…" is gone:

// Runs any command: execute() decides first whether it needs the user's approval.

Run it

Still from a throwaway directory, still in a real terminal. Guardrails reduce the risk, they don't remove it (more on that below):

cp -R /path/to/some/project /tmp/to-be-trashed-after && cd /tmp/to-be-trashed-after
export LLM_API_KEY=...
jbang /path/to/build-your-harness/06-guardrails/Agent.java --inspect

When a command needs your approval, the header shows [approval needed]: y approves, n or Esc refuses.

Try it

Let's start with the usual:

Explore this project and summarize it.

ls, cat, head… run without a single question. Then, most likely, a find asks for your approval. Why? find looks read-only, but find . -delete or find … -exec rm {} \; can change anything. So it's not in the list. Approve it: you'll see the result, and the agent goes on, almost as smooth as in article 05.

Now something less innocent:

Clean up the temporary files of this project.

The full rm … command shows up, with Allow? [y]es / [n]o, before anything is deleted. Answer n. You see ✗ denied, and the model gets the refusal as its tool result. Then it adapts: it asks you which files you want removed, or explains what it was about to do.

Chaining, the friendly way:

List the files, then show the start of the README, in one command.

ls && head README.md runs without asking: every part is read-only.

Chaining, the other way:

Delete the build directory and list what's left, in one command.

rm -rf build && ls asks. One part is enough.

And a harmless-looking one:

Which Java version is installed?

java -version asks. Why? Because java is not in the list, and it shouldn't be: java can run any code. Our allowlist is about programs that can't change anything, not commands that look harmless.

If you want to see the round limit, lower MAX_TOOL_ROUNDS to 2 and ask for the project summary again. The loop stops with "Stopped: more than 2 tool rounds for one request."

Approving blindly

One last experiment. Ask:

Add a header comment to Agent.java.

(or any file of your throwaway project). The model has no tool to edit files, so it uses the shell. You get something like sed -i '1i // ...' Agent.java, or a cat > file <<EOF with the whole file inside. And you have to approve it.

Can you tell, from that sed, exactly what the file will look like afterwards? Will it break something? With a one-liner, maybe. With a real edit in the middle of a file, a regular expression, escaped characters… no way 😵 So you approve blindly, or you refuse everything. Neither is what we want from a guardrail.

The approval works. What we approve is the problem: a raw shell command says how to change a file, not what changes.

Limits

• An allowlist is a trade-off, not a guarantee. git diff --output=file writes a file, and it passes our check. cat ~/.ssh/id_rsa is perfectly "read-only"… and sends your private key to the model provider. Read-only doesn't mean harmless.
• Models love 2>/dev/null. The > makes those commands ask, even when they're read-only. Safe, a bit annoying.
• Approving a shell command is not knowing what it changes, as we've just seen.
• The real defense is isolation: a container, a sandbox, a dedicated user with access to nothing important. Guardrails in the harness reduce the risk, they don't remove it. Look at the agents you use every day: they all have an equivalent of this allowlist, and they all recommend a sandbox.

As usual, the code, a README to run it and things to try are in the step-06 folder.

Next time

How do you approve a change you can't see? Next article: dedicated tools to write and edit files, each with its own permission, and an edit you approve as a diff before it is written.

Tell me what you think 💞