# Harness Engineering Harness Engineering is the discipline of designing constraints, tools, feedback loops, documentation, and verification systems that guide AI agents toward reliable, maintainable outputs. Where [[Context Engineering]] focuses on *what information reaches the model*, and [[Prompt Engineering]] focuses on *how you phrase the question*, harness engineering focuses on *how the agent runs*: the environment, guardrails, and feedback mechanisms that surround its execution. The term was coined in late 2025 and formalized in early 2026, notably through OpenAI's internal experiments with [[OpenAI Codex|Codex]], where they built a production application with over 1 million lines of code written entirely by agents. The key insight: the agent wasn't the hard part. The harness was. ## Core components According to the OpenAI model (via Birgitta Böckeler's analysis on Martin Fowler's site), a harness operates across three layers: 1. **Context Engineering**: the foundational layer. Continuous enhancement of knowledge bases embedded in codebases, supplemented by agent access to dynamic information sources. This is where [[Context Engineering]] sits *within* the harness 2. **Architectural Constraints**: enforcement mechanisms combining LLM-based monitoring with deterministic custom linters and structural tests. These constrain the agent's solution space 3. **Periodic Maintenance ("Garbage Collection")**: agents that regularly scan for documentation inconsistencies and architectural violations to combat entropy and code decay ## Relationship to other disciplines | Discipline | Focus | Core Question | |---|---|---| | [[Prompt Engineering]] | Wording and phrasing | "What should I say?" | | [[Context Engineering]] | Information environment | "What information is relevant?" | | **Harness Engineering** | Execution environment | "How should the agent run?" | | [[Intent Engineering]] | Goals and outcomes | "What must be accomplished?" | | [[Agent System Engineering]] | Multi-agent systems | "How do agents work together?" | Harness engineering builds on prompt and context engineering but shifts to execution optimization. It addresses the core limitation of pure context engineering: agents still drift, accumulate entropy, and fail in long-running tasks without structured guardrails. ## Key principles - **Constrain the solution space**: reliability requires limiting flexibility through standardized patterns, not unrestricted generation - **Iterative refinement**: when agents struggle, gaps in documentation, guardrails, or tools become signals for improvement - **Long-term maintainability**: emphasis on internal quality preservation over short-term velocity - **Feedback loops**: closed-loop failure tracking and pattern clustering to systematically improve agent behavior - **Hybrid verification**: combine deterministic tools (linters, type checkers) with AI-driven validation - **Turn repeated corrections into checks**: if you have to correct the same thing twice, it belongs in a test or a linter, not in another line of instructions ## Practical examples - [[Claude Code]] with CLAUDE.md files, skills, hooks, and [[Model Context Protocol (MCP)|MCP]] servers providing task-specific scaffolding - [[AI Agent Harness]] implementations like Cursor, Cline, Aider - CI/CD pipelines that validate agent-generated code before merge - Architectural decision records that constrain agent design choices - Custom harnesses built on an SDK such as the [[Pi Mono|Pi]] SDK (see below) ## What the Hacker News discussion added (July 2026) In July 2026, Ryan Lopopolo (the author of OpenAI's harness engineering post) published a GitHub repo he calls an "anthology, field guide, and agent context bundle" on the topic, and answered questions about it on [[Hacker News]]. The repo frames it like this: keep the model and the coding agent constant and treat them as a black box. Then improve the two levers you control around them, context and tools. The harness is where an organization's nonfunctional requirements live (reliability, security, performance, maintainability, polish), and every accepted change, correction or failure should leave behind context, tools or checks that make the next run better. One commenter asked how to keep naming coherent when the agent calls the same concept a "Materialization" one day and a "Projection" the next. Lopopolo's answer: as soon as the same drift shows up more than once, stop and have the agent write a static verifier that rejects everything except the option you want. His examples were an AST-walking linter that fails when a variable or parameter named like `...Ms` or `...Sec` is typed as a plain `number`, and a rule forcing Zod schemas into `ZPascalCase` with matching `PascalCase` types. He calls this a form of recursive self-improvement, and prefers it to fine-tuning for a simple reason: the checks survive model upgrades. You adopt the next model release and keep everything you've encoded about your local acceptance criteria. He also described one-shotting a DICOM image viewer in about 30 minutes. It ran at 60% CPU because it didn't cache decoded frames. Fixing that one bug is easy. The question he cared about was why the harness let a "done" artifact ship with that problem, and how to make that whole class of performance mistake fail the acceptance criteria automatically. On cost, his point was that cheap models still need solid tool calling, but they can go further than you'd think if bigger models write the guardrails first: once static verifiers exist, a cheaper model can steer itself as the tests fail. One commenter reported DeepSeek V4 Flash as good enough at tool calling, at about $1.50 for 40 million tokens. One commenter summed up agentic engineering as "good software engineering practices enforced adversarially". Another pointed out that accountants, shop owners and electrical engineers are building personal harnesses around Claude and Codex without ever having heard the word "harness". Plenty of commenters were skeptical. The README opens with a quote claiming agents get "100x" better output by reading the author's work, and nobody could say where that number comes from. Lopopolo confirmed that the entire repo was written by GPT-5.6 Sol Ultra (he says he read and approved every word), and several readers found it hard to read. One commenter called the idea of pointing your agent at someone else's instruction repo "the mother of all prompt injections". Others saw yet another buzzword after prompt and context engineering. Nobody answered the most useful question in the thread: how do you verify that a harness actually improves results? ## Building your own harness Off-the-shelf harnesses like [[Claude Code]] or [[Codex CLI]] come with the vendor's rules built in. A custom harness is your own agent loop, and it moves three decisions into your code: - Routing: which model handles this request - Gating: is this tool call allowed? There are three possible answers (allow, ask a human, block), so you need two thresholds, not one - Verification: is the answer complete and grounded in what the agent actually saw, before it goes back to the user Each of those checks used to cost a full LLM call, so most harnesses skipped them. [[Decision Models (DMs)|Decision models]] such as [[Jev]] change the math: a check takes roughly 200 to 400 ms and costs a fraction of a cent, which is cheap enough to run on every step. Not every check needs a model. Deterministic ones (is this path inside the project folder?) stay in plain code. Only the probabilistic ones (does this shell command look destructive? does the answer cover the whole question?) go to the decision model. A few design rules are worth deciding before you write any code: - Failure modes: when the decision model is down or times out, the gate fails closed (block the call) and the router fails open (use the strong model) - Model choice: pick the model once per request. Switching models in the middle of a session breaks prompt caching; the tutorial below cites about 50% higher cost when one long session goes from Opus to Sonnet and back to Opus - Logging: write every decision to a log (`decisions.jsonl`), so you can tune the thresholds on real traffic instead of guessing For a concrete example, Elvis Saravia (DAIR.AI) wrote a tutorial that builds exactly this on the [[Pi Mono|Pi]] SDK with Jev. It was inspired by Sydney Runkle's "Building a Harness with Jev" on the [[LangChain]] blog, which ships routing and gating as ready-made middleware. Pi 1.0 also covers part of it natively: virtual models (an extension picks a physical model for each request) and built-in classifier models, Jev included. I'm exploring a custom harness for my [[Obsidian Starter Kit]] vault. Today, the rules for my agents are written as prose in AGENTS.md; in my own harness, many of them would become code. ## References - https://openai.com/index/harness-engineering/ - https://martinfowler.com/articles/exploring-gen-ai/harness-engineering.html - https://blog.langchain.com/the-anatomy-of-an-agent-harness/ - https://www.agent-engineering.dev/article/harness-engineering-in-2026-the-discipline-that-makes-ai-agents-production-ready - Hacker News discussion (July 2026): https://news.ycombinator.com/item?id=48963483 - Ryan Lopopolo's harness engineering repo: https://github.com/lopopolo/harness-engineering - Building a Custom Harness with Pi and Jev (Elvis Saravia, DAIR.AI): https://academy.dair.ai/resources/jev-decisions-in-a-pi-sdk-harness - Building a Harness with Jev (Sydney Runkle, LangChain): https://www.langchain.com/blog/building-a-harness-with-jev - Pi: https://github.com/earendil-works/pi ## Related - [[Context Engineering]] - [[Prompt Engineering]] - [[Intent Engineering]] - [[Agent System Engineering]] - [[AI Agent Harness]] - [[AI Agents]] - [[AI Agent Skills]] - [[AI Agent Orchestration]] - [[Claude Code]] - [[How coding agents work]] - [[Agentic Engineering]] - [[Feedback Loop]] - [[Levels of AI use]] - [[SOLID Principles]] - [[Software Design Patterns for AI Skills and Agents]] - [[Loop Engineering]] - [[Living shim between humans and AI models]] - [[Pi Mono]] - [[Decision Models (DMs)]] - [[Jev]]