Build your own agent harness, part 1: the model doesn't remember you

a panda building a harness for its AI agents

Every day I use coding agents. Claude Code, Codex, Kimi Code, you name it. They read my files, run my tests, fix my typos (and sometimes add a few of their own 😇). And at some point the question came back, the same one I had at 20 when I was compiling kernels just to see how they worked: what is actually inside?

Spoiler: a model only completes text. That's it. What turns it into an agent is everything around it: the conversation history, the loop, the tools, the guardrails, the context management. That's what we call the harness.

So let's build one. In Java, yes (I told you last time, Bye bye mkdocs, bonjour Roq 😜). One concept per article, every part of the harness written by hand, and at each step we look at what really goes over the wire. By the end, you'll know what's inside the agents you use every day, because you'll have written one.

The plan for this series

A few choices before we start, so nobody is surprised:

Java 25 compact source files + JBang: each step is one runnable file, no build project, no pom.xml (I can hear some of you cheering 😇).
LangChain4j, low-level API only: it handles HTTP, JSON and the provider format, and nothing else. The history, the loop, the tools: all ours.
The OpenAI Responses API: any endpoint implementing it will do.
--inspect: a flag that dumps the raw JSON requests and responses to a file. It's our "no magic" proof, and we'll use it in every article.

The code lives in a companion repository, build-your-harness, with one tag per article. This one is step-01.

Today we build the smallest possible harness: a history and a loop. And on the way, we'll discover the first truth about LLMs.

The model doesn't remember you

When you chat with a model, it really feels like a conversation. You say something, it answers, you go on, it follows. It remembers.

Except it doesn't. The model is stateless. Each request is a brand new start: it has no idea what you said a minute ago. If it seems to remember, it's because someone sent it the whole conversation again. That someone is the harness. Today, that's us.

What we build

First, the model. Here is how we create it in Agent.java:

ChatModel model = OpenAiResponsesChatModel.builder()
        .baseUrl(BASE_URL).apiKey(API_KEY).modelName(MODEL)
        .store(false) // stay stateless: the server keeps nothing, we resend the history
        .logRequests(inspect).logResponses(inspect) // raw JSON on the wire, written to inspect.log
        .build();

Any endpoint implementing the Responses API, configured through three environment variables: LLM_BASE_URL, LLM_API_KEY and LLM_MODEL (by default the OpenAI URL and gpt-6-luna). Keep an eye on that store(false), we'll come back to it.

And now, the whole harness. Yes, the whole thing:

// The model is stateless: this list IS the conversation, and we resend all of it every turn.
List<ChatMessage> history = new ArrayList<>();

display.banner(MODEL, BASE_URL);
while (true) {
    String input = display.readUserInput();
    if (input == null || input.equals("exit")) break;
    history.add(UserMessage.from(input));
    AiMessage answer = model.chat(history).aiMessage();
    history.add(answer);
    display.answer(answer.text());
}

A list and a loop. Each turn, we add the user message to the list, we send the whole list to the model with model.chat(history), and we add the answer to the list. That List<ChatMessage> is the conversation. Everything in the rest of this series builds on it.

A few details worth a sentence:

display is the console rendering, in common/Display.java. I kept it out of the step file on purpose: Agent.java only shows harness logic.
The header lines at the top of the file (//DEPS, //SOURCES, //FILES) are JBang directives: dependencies and extra files, no build tool needed.
void main() without any class around it? That's a Java 25 compact source file (JEP 512). Java has changed, I promise.

Run it

export LLM_API_KEY=...
jbang 01-chat/Agent.java --inspect

And in a second terminal, to watch what goes over the wire:

tail -F inspect.log

The experiment

Time for the "aha" moment. Let's introduce ourselves:

you> My name is Alice.

assistant> Nice to meet you, Alice!

you> What is my name?

assistant> Your name is Alice.

The model answers correctly. Of course it does, you'd say. But look at inspect.log. Here is the body of the second request (I only shortened the encrypted blob):

{ "model" : "gpt-6-luna", 
"input" : [ 
  { "type" : "message", "role" : "user", "content" : [ { "type" : "input_text", "text" : "My name is Alice." } ] }, 
  { "type" : "reasoning", "encrypted_content" : "gAAAAABq213vMO6YnnProbhsXNKNSggQSswhwBpMfwGd87R97vmZ…", "summary" : [ ] }, 
  { "type" : "message", "role" : "assistant", "content" : [ { "type" : "output_text", "text" : "Nice to meet you, Alice!" } ] }, 
  { "type" : "message", "role" : "user", "content" : [ { "type" : "input_text", "text" : "What is my name?" } ]} ],
  "stream" : false,
  "store" : false
}

My first message, the model's answer, my new question. That's the "memory". Nothing more, nothing less: we sent it everything again.

(Yes, there's a fourth item, a reasoning one with an unreadable encrypted_content. gpt-6-luna is a reasoning model, and its encrypted thinking from the previous turn travels back with the rest of the history. LangChain4j handles it for us here; we'll come back to it when it starts to matter.)

Now let's be a bit mean. On line 38, instead of the whole history, we only send the current message:

AiMessage answer = model.chat(List.of(UserMessage.from(input))).aiMessage();

Run it again, same two prompts:

you> My name is Alice.

assistant> Nice to meet you, Alice!

you> What is my name?

assistant> I don’t know—you haven’t told me your name.

The model didn't "forget" anything: it was never told. Conversation memory is entirely the harness's job. Mind blown? A little? Come on 😉

We could let the server remember… and we refuse

Fair point: the Responses API can keep the conversation for us. With store enabled, the server saves each response, and you pass a previous_response_id to continue from it. No list to resend, less JSON on the wire. Tempting.

We say no, on purpose, with store(false). If the server holds the history, we lose control of what's in the context. And deciding what goes into the context is precisely what a harness does, as we'll see in later articles. So the history stays here, in our list, where we can see it and touch it.

What the library does and doesn't do

LangChain4j's low-level ChatModel takes our list of messages and turns it into HTTP + JSON in the provider's format. That's its job, and I'm happy not to write that plumbing myself. w What it does not do is keep the history: our code does. LangChain4j also has a high-level API, AiServices, which would hide that list from us. Very handy in real life, but here it would hide exactly what we want to understand, so we stay at the low level. (We'll come back to it at the very end of the series.)

Limits

Let's be honest, this is a small thing:

The history grows forever: every turn resends everything said so far.
No system prompt, no tools: it's a chat, not an agent yet.
It needs an endpoint implementing the OpenAI Responses API, not only Chat Completions.

The code, a README to run it and things to try are all in the step-01 folder.

Next time

Every turn resends the whole history. So what does a simple "ok" actually cost? Next article: a system prompt, tokens, cost, and how full the context window gets.

Tell me what you think 💞