# Harness Engineering Harness Engineering is the discipline of designing constraints, tools, feedback loops, documentation, and verification systems that guide AI agents toward reliable, maintainable outputs. Where [[Context Engineering]] focuses on *what information reaches the model*, and [[Prompt Engineering]] focuses on *how you phrase the question*, harness engineering focuses on *how the agent runs*: the environment, guardrails, and feedback mechanisms that surround its execution. The term was coined in late 2025 and formalized in early 2026, notably through OpenAI's internal experiments with [[OpenAI Codex|Codex]], where they built a production application with over 1 million lines of code written entirely by agents. The key insight: the agent wasn't the hard part. The harness was. ## Core components According to the OpenAI model (via Birgitta Böckeler's analysis on Martin Fowler's site), a harness operates across three layers: 1. **Context Engineering**: the foundational layer. Continuous enhancement of knowledge bases embedded in codebases, supplemented by agent access to dynamic information sources. This is where [[Context Engineering]] sits *within* the harness 2. **Architectural Constraints**: enforcement mechanisms combining LLM-based monitoring with deterministic custom linters and structural tests. These constrain the agent's solution space 3. **Periodic Maintenance ("Garbage Collection")**: agents that regularly scan for documentation inconsistencies and architectural violations to combat entropy and code decay ## Relationship to other disciplines | Discipline | Focus | Core Question | |---|---|---| | [[Prompt Engineering]] | Wording and phrasing | "What should I say?" | | [[Context Engineering]] | Information environment | "What information is relevant?" | | **Harness Engineering** | Execution environment | "How should the agent run?" | | [[Intent Engineering]] | Goals and outcomes | "What must be accomplished?" | | [[Agent System Engineering]] | Multi-agent systems | "How do agents work together?" | Harness engineering builds on prompt and context engineering but shifts to execution optimization. It addresses the core limitation of pure context engineering: agents still drift, accumulate entropy, and fail in long-running tasks without structured guardrails. ## Key principles - **Constrain the solution space**: reliability requires limiting flexibility through standardized patterns, not unrestricted generation - **Iterative refinement**: when agents struggle, gaps in documentation, guardrails, or tools become signals for improvement - **Long-term maintainability**: emphasis on internal quality preservation over short-term velocity - **Feedback loops**: closed-loop failure tracking and pattern clustering to systematically improve agent behavior - **Hybrid verification**: combine deterministic tools (linters, type checkers) with AI-driven validation - **Turn repeated corrections into checks**: if you have to correct the same thing twice, it belongs in a test or a linter, not in another line of instructions - **Watch for shouting in your prompts**: when you repeat a rule or write it in ALL CAPS, you're probably [[Fighting the Weights]]. That rule is a candidate for a hook, a linter or a better tool interface ## Practical examples - [[Claude Code]] with CLAUDE.md files, skills, hooks, and [[Model Context Protocol (MCP)|MCP]] servers providing task-specific scaffolding - [[AI Agent Harness]] implementations like Cursor, Cline, Aider - CI/CD pipelines that validate agent-generated code before merge - Architectural decision records that constrain agent design choices - Custom harnesses built on an SDK such as the [[Pi Mono|Pi]] SDK (see below) ## What the Hacker News discussion added (July 2026) In July 2026, Ryan Lopopolo (the author of OpenAI's harness engineering post) published a GitHub repo he calls an "anthology, field guide, and agent context bundle" on the topic, and answered questions about it on [[Hacker News]]. The repo frames it like this: keep the model and the coding agent constant and treat them as a black box. Then improve the two levers you control around them, context and tools. The harness is where an organization's nonfunctional requirements live (reliability, security, performance, maintainability, polish), and every accepted change, correction or failure should leave behind context, tools or checks that make the next run better. One commenter asked how to keep naming coherent when the agent calls the same concept a "Materialization" one day and a "Projection" the next. Lopopolo's answer: as soon as the same drift shows up more than once, stop and have the agent write a static verifier that rejects everything except the option you want. His examples were an AST-walking linter that fails when a variable or parameter named like `...Ms` or `...Sec` is typed as a plain `number`, and a rule forcing Zod schemas into `ZPascalCase` with matching `PascalCase` types. He calls this a form of recursive self-improvement, and prefers it to fine-tuning for a simple reason: the checks survive model upgrades. You adopt the next model release and keep everything you've encoded about your local acceptance criteria. He also described one-shotting a DICOM image viewer in about 30 minutes. It ran at 60% CPU because it didn't cache decoded frames. Fixing that one bug is easy. The question he cared about was why the harness let a "done" artifact ship with that problem, and how to make that whole class of performance mistake fail the acceptance criteria automatically. On cost, his point was that cheap models still need solid tool calling, but they can go further than you'd think if bigger models write the guardrails first: once static verifiers exist, a cheaper model can steer itself as the tests fail. One commenter reported DeepSeek V4 Flash as good enough at tool calling, at about $1.50 for 40 million tokens. One commenter summed up agentic engineering as "good software engineering practices enforced adversarially". Another pointed out that accountants, shop owners and electrical engineers are building personal harnesses around Claude and Codex without ever having heard the word "harness". Plenty of commenters were skeptical. The README opens with a quote claiming agents get "100x" better output by reading the author's work, and nobody could say where that number comes from. Lopopolo confirmed that the entire repo was written by GPT-5.6 Sol Ultra (he says he read and approved every word), and several readers found it hard to read. One commenter called the idea of pointing your agent at someone else's instruction repo "the mother of all prompt injections". Others saw yet another buzzword after prompt and context engineering. Nobody answered the most useful question in the thread: how do you verify that a harness actually improves results? (Google Research's [[Harness2|Harness²]], published in October 2026, offers one answer: run competing harness edits on the same task and compare how they behave.) ## The harness is the body (October 2026) In October 2026, another harness thread reached the front page of [[Hacker News]]: "Harness Engineering 101", a free book by Isuru Wijesiri. Its one-line pitch is the clearest beginner explanation I've seen so far: "The LLM is the brain. The harness is the body." The book starts from first principles. A model API is stateless. On every turn, you send the whole conversation as a JSON array and you get text back. That's it. The harness is the program that builds, maintains and protects that array. Everything else we call an "agent" is a patch on that one loop ([[Agentic loops]]), and every patch exists because something concrete broke. Each chapter follows the same shape: the failure, the smallest patch that fixes it, and what the patch costs. Across 16 chapters, a toy agent grows to about 300 lines of plain Python. A few chapter titles are worth keeping on their own, because they compress a whole idea: - **"Context Is a Budget, Not a Bag."** The [[Context Window]] is something you spend, not a place where you dump things. That's [[Context Engineering]] in five words - **"Every Framework Is a Wrapper Around Chapter 1."** Chapter 1 is the JSON array. LangChain, the agent SDKs, [[Pi Mono|Pi]]: they all build and send that array for you - **"You can't fix the prompt you can't see."** Log the exact array you send. Most agent bugs become obvious once you read what the model actually received - **"RAG Was a Harness Pattern All Along."** [[Retrieval-Augmented Generation (RAG)|RAG]] is just the harness putting search results in the array before the model call - **Caching: why order matters.** Put the stable parts (system prompt, tools) first and the changing parts last, or you pay full price on every turn The other chapters cover [[AI Subagents]] (forking the context), steering a running loop, background work, [[Model Context Protocol (MCP)|MCP]], [[AI Agent Skills|skills]], deferred loading, [[Claude Code Hooks|hooks]], guardrails, and model routing. The comments added a few points I agree with: - **Build one to learn.** Several people recommended writing a minimal harness from scratch, starting from Thorsten Ball's post "How to build an agent", with a local tool-calling model like [[Qwen]]. Let a frontier model write the boilerplate, but write the core loop yourself - **Smart model for one-off tasks, cheap model for repeated ones.** For a one-off, use the best model with a powerful harness. For work you repeat, move the know-how into tools and instructions, then use the cheapest model that still works. Same idea as Lopopolo's verifiers above - **The harness can hold as much intelligence as the model.** Memory, planning, routing between models, even classical machine learning. Sometimes you don't need an LLM at all for a step (see [[Machine Native Intelligence]] and [[Decision Models (DMs)|decision models]]) - **Small and bounded beats big and vague.** A solo developer built a file-editing harness in under a day. The main benefit, in their words: they know exactly where its boundaries are - **Security is still mostly DIY.** One commenter runs the agent in a container with no credentials, sends authenticated requests through a gateway, and only allows a list of approved domains through the firewall. They found little good guidance on this, and I've had the same experience The skeptics asked whether smarter models will make harnesses unnecessary. I don't think so. Better models removed some patches (we write far less prompt scaffolding than in 2024), but the array still has to be built, the costs still have to be controlled, and your own acceptance criteria still have to live somewhere. The model gets smarter. The body still decides what it can touch. Some readers also complained (again) that the prose felt AI-polished; the author disclosed using Claude to polish the English. And many found the analogies (kitchens, chefs, climbers) overdone. Fair, but the brain and body one works. ## Building your own harness Off-the-shelf harnesses like [[Claude Code]] or [[Codex CLI]] come with the vendor's rules built in. A custom harness is your own agent loop, and it moves three decisions into your code: - Routing: which model handles this request - Gating: is this tool call allowed? There are three possible answers (allow, ask a human, block), so you need two thresholds, not one - Verification: is the answer complete and grounded in what the agent actually saw, before it goes back to the user Each of those checks used to cost a full LLM call, so most harnesses skipped them. [[Decision Models (DMs)|Decision models]] such as [[Jev]] change the math: a check takes roughly 200 to 400 ms and costs a fraction of a cent, which is cheap enough to run on every step. Not every check needs a model. Deterministic ones (is this path inside the project folder?) stay in plain code. Only the probabilistic ones (does this shell command look destructive? does the answer cover the whole question?) go to the decision model. A few design rules are worth deciding before you write any code: - Failure modes: when the decision model is down or times out, the gate fails closed (block the call) and the router fails open (use the strong model) - Model choice: pick the model once per request. Switching models in the middle of a session breaks prompt caching; the tutorial below cites about 50% higher cost when one long session goes from Opus to Sonnet and back to Opus - Logging: write every decision to a log (`decisions.jsonl`), so you can tune the thresholds on real traffic instead of guessing For a concrete example, Elvis Saravia (DAIR.AI) wrote a tutorial that builds exactly this on the [[Pi Mono|Pi]] SDK with Jev. It was inspired by Sydney Runkle's "Building a Harness with Jev" on the [[LangChain]] blog, which ships routing and gating as ready-made middleware. Pi 1.0 also covers part of it natively: virtual models (an extension picks a physical model for each request) and built-in classifier models, Jev included. I'm exploring a custom harness for my [[Obsidian Starter Kit]] vault. Today, the rules for my agents are written as prose in AGENTS.md; in my own harness, many of them would become code. ## References - https://openai.com/index/harness-engineering/ - https://martinfowler.com/articles/exploring-gen-ai/harness-engineering.html - https://blog.langchain.com/the-anatomy-of-an-agent-harness/ - https://www.agent-engineering.dev/article/harness-engineering-in-2026-the-discipline-that-makes-ai-agents-production-ready - Hacker News discussion (July 2026): https://news.ycombinator.com/item?id=48963483 - Ryan Lopopolo's harness engineering repo: https://github.com/lopopolo/harness-engineering - Building a Custom Harness with Pi and Jev (Elvis Saravia, DAIR.AI): https://academy.dair.ai/resources/jev-decisions-in-a-pi-sdk-harness - Building a Harness with Jev (Sydney Runkle, LangChain): https://www.langchain.com/blog/building-a-harness-with-jev - Pi: https://github.com/earendil-works/pi - Harness Engineering 101 (Isuru Wijesiri, October 2026): https://github.com/IsuruMaduranga/harness-engineering-101 - Hacker News discussion of Harness Engineering 101 (October 2026): https://news.ycombinator.com/item?id=50151390 ## Related - [[Harness2]] - [[Context Engineering]] - [[Prompt Engineering]] - [[Intent Engineering]] - [[Agent System Engineering]] - [[AI Agent Harness]] - [[AI Agents]] - [[AI Agent Skills]] - [[AI Agent Orchestration]] - [[Claude Code]] - [[How coding agents work]] - [[Agentic Engineering]] - [[Feedback Loop]] - [[Levels of AI use]] - [[SOLID Principles]] - [[Software Design Patterns for AI Skills and Agents]] - [[Loop Engineering]] - [[Living shim between humans and AI models]] - [[Pi Mono]] - [[Decision Models (DMs)]] - [[Jev]] - [[Agentic loops]] - [[Context Window]] - [[Retrieval-Augmented Generation (RAG)]] - [[AI Subagents]] - [[Machine Native Intelligence]] - [[Fighting the Weights]]