Agent Harness Configuration
Managing agents is easy using…
AGENTS.md…
and agent skill files…
and using subagents…
and MCP servers…
and plugins…
and rule files…
and agent specs…
and hooks…
and settings…
and an LLM wiki…
But just in case it’s useful, I threw together a bit of a cheat sheet here to help keep it all sorted.
I’ll be following this up with more AI-related blog posts every day for a while. And although I use AI for everything these days, none of these posts will be generated or written by AI. So if you find this cheat sheet useful, follow along!
AGENTS.md (also CLAUDE.md and GEMINI.md)
The de facto starting place to begin building your guardrails. You can reasonably start with this and “refactor” into the other tools as your needs grow.
- ADOPT: Use these files for context that can’t be inferred from the code; this is a README.md file for agents.
- TRIAL: Use nested/layered files to keep context close to the code and services to which they apply. This will become a best practice over the next year but is not fully supported yet by all vendors.
- TRIAL: Point CLAUDE.md and GEMINI.md to AGENTS.md
@AGENTS.mdto enable instruction across vendor platforms. I’ve marked this as trial because if using nested/layered AGENTS.md files, creating the companion instruction files with@AGENTS.mdmay or may not work depending on the tooling. - CAUTION: Be wary of instruction bloat or trying to invent expertise. (“You are an expert in…”). This is a place for execution policy but not accumulated knowledge.
Agent Skills
Skills are a great way to improve the consistency of agent behavior while at the same time reducing context bloat by providing procedural knowledge and workflows to agents, but only when needed.
Skills aren’t a governed standard yet like MCP and AGENTS.md, but it’s an open standard that’s not going anywhere.
- ADOPT: If you find yourself creating checklists or describing processes in AGENTS.md, move them to an agent skill instead.
- ADOPT: Agent skills support the inclusion of reference documents and scripts. I don’t see enough people using scripts with their skills to increase the determinism of their workflows.
- TRIAL: Replace custom prompt files. If using custom prompt files as with GitHub Copilot, begin replacing these with skills to give cross-vendor support and to take advantage of increased functionality like scripts and references. Custom prompt files are already being deprecated by tools like OpenAI Codex and Claude Code.
- CAUTION: Correct skill loading isn’t a guarantee and depends on both model performance and well-written skill descriptions. Writing good descriptions is essential. Prefer hooks when you need to guarantee that a workflow needs to happen and you can pin it to a lifecycle event.
Subagents
I like subagents a lot, but the implementation varies widely across vendors. For example, some subagents can invoke their own subagents. With some tools, subagents can work collaboratively (for example with Claude Code and agent teams enabled). This is an emerging pattern.
Subagents are/will be great for reducing context in the orchestrator agent, mitigating “reasoning over long context” challenges, and maintaining agent alignment through the use of agent-specific content. Adversarial code review agents are an example of the latter, and I’ll be doing a blog post about those soon.
- CAUTION: Definitions aren’t standard across providers. If enabling a repository for multiple tools, be sure to check the documentation for each to include supported configuration properties.
- CAUTION: Don’t rely on experimental orchestration patterns. I haven’t yet seen great results from Claude agent teams, for example, which is probably why it’s still in experimental status. However, new orchestration types will likely continue to become available in harness tooling and eventually lead to great use cases.
MCP
Created by Anthropic and one of the first governed AI specs. Use MCP servers to make external context, tools, and data sources available to your agents.
- ADOPT: Use MCP servers to ground your agent sessions, for example by having agents look up version-specific documentation for an API or SDK before writing code.
- TRIAL: “The CLI is all you need.” One way to avoid an excessive amount of tools is to simply prefer terminal use instead of an MCP server, especially with platforms like Claude Code that allow you to configure which commands are allowed, denied, or should prompt for user approval. (Unrestricted CLI access would of course be inherently insecure.) This can be powerful when combined with guardrails like service principals with limited/restricted access, etc.
- CAUTION: Don’t enable too many tools at once. This causes context bloat and has been formally shown in research and benchmarks to reduce performance.
- CAUTION: Be wary of tool poisoning attacks in MCP servers that can lead to problems like data exfiltration.
Plugins
This is essentially a packaging layer for harnesses. And while there is an “Open Plugin Specification v1.0.0” from Vercel, the implementation of plugins still varies widely across vendors who use overlapping concepts but different manifests, marketplace mechanics, and permissions.
Open Codex: let’s use bundle skills, app integrations, and MCP servers. Claude Code: plugins are a self-contained directory that can include skills, agents, hooks, MCP servers, and more.
- TRIAL: Enterprises should experiment with using plugins as a mechanism for internal-only distribution of tooling. (Codex and Claude both support this.)
- TRIAL: Convenient for individual developers using just one or two tools.
- CAUTION: Varies widely across vendors.
Spec-driven Development and References
Spec-first, spec-anchored, and spec-as-source, these are techniques for using additional context and implementation details to drive agentic development.
Specs can be great for steering development and making your intent clear to the model. But they require attention in the creation of said specs. If you vibe code your specs, you might as well just have skipped creating them in the first place, although they can improve consistency at least.
- ADOPT: Use documents like ADRs (architectural decision records) to help prevent drift. Ensure that the existence of these documents is noted in the top-level AGENTS.md file but allow agents to read those documents as needed.
- ADOPT: When building out ADRs, use a status field so that agents know whether a doc is approved and should be enforced or is just proposed. You might use proposed a lot while still in the planning stages of a greenfield project to prevent model rigidity.
- CAUTION: Agents aren’t forced to read specs in the same way that they are a root-level AGENTS.md file. Therefore, be sure to point to your specs in said file, the appropriate skill files, and so on.
- CAUTION: You could enforce the reading of specs with hooks, but that will cause context bloat. Better is to have a review subagent ensure that specs were followed.
Hooks
Hooks are triggers that let you execute commands and prompts at specific lifecycle events. For example, one way you could implement a Ralph-Wiggum loop is to use a stop hook to determine if any PBIs or tasks in a spec file remain unfinished and to continue execution until all tasks are complete. (You could also do this in a script yourself.)
- ADOPT: Use hooks to run linters and formatters.
- TRIAL: Use hooks to increase determinism in your workflows.
- CAUTION: Implementations and lifecycles vary between vendors. Hooks need to be written and tested for each platform you’re using. (Codex, Claude, GitHub Copilot, etc.)
- CAUTION: Don’t assume hooks will always fire. Support may not have reached general availability for some tools and platforms. For example, with OpenAI Codex they were disabled on Windows systems while working on Mac.
Rule Files
Some platforms have the concept of a “rules file” that allows you to load context and instructions based on matching patterns in the frontmatter of a markdown file. For example, from the Claude Code documentation:
---
paths:
- "**/*.test.ts"
- "**/*.test.tsx"
---
# Testing Rules
- Use descriptive test names: "should [expected] when [condition]"
- Mock external dependencies, not internal modules
- Clean up side effects in afterEach
There is some overlap here with nested/layered AGENTS.md files and skill files. And this isn’t supported across all platforms.
- TRIAL: Use rule files for path- or filetype-specific instructions when you don’t need cross-platform support and/or where layered AGENTS.md files aren’t supported. These will more deterministically match than using a skill description or pointing to a folder of specs from an AGENTS.md, which the agent may or may not subsequently read.
- TRIAL: Use rule files to keep filetype-specific instructions out of general context. For example, for
**/*.pyfiles you might specify “Always use type hints.” But we would not need that instruction for our front-end JavaScript.
Note: I’ve marked these as trial because it’s not clear to me that they always work consistently with subagents.
Settings/Configuration
The most deterministic thing we can do with AI is configure our tools, their access, and their sandbox settings.
- ADOPT: Enforce what agents are allowed to do and the commands they are allowed to run.
- TRIAL: Combine settings with containerized environments to minimize blast radius when running with more permissive settings. However, be careful of the possibility of data exfiltration with project injection attacks.
- CAUTION: Sandboxes and general configuration vary widely from platform to platform.
LLM-Wiki
The concept of an LLM Wiki was proposed by Andrej Karpathy in a tweet back around March. The idea is to reduce agents repeating the same discovery with each new session and create/maintain an evolving knowledge layer that improves model/agent comprehension. It differs from static context found in files like AGENTS.md in that the knowledge layer can grow and expand in a dynamic and interconnected way with the hope of making relationships and insights available to models without them having to reinvent general relativity every new session.
- CAUTION: I like this idea and I’ve been using it myself. However, I’ve had mixed results. You lose some determinism with the information in the wiki, so you have to be careful to make sure that important information is located in places like AGENTS.md and skill files. I suspect this is best for large and complicated systems, and I would advise against it for anything simpler for now.
More to Come!
That’s it for now. I’ll be writing deep-dive posts for much of this, including other topics like how to more consistently create images that look less like AI slop.
By the way, I’m looking for my next opportunity, and I’m available for consulting. Shoot me an email, and let’s get on a call and talk!