Home
Jul 8, 2026
View All

Harness Tuning, Agent Infrastructure, and What's Actually Shipping

The Short Version#

The theme today is agent infrastructure maturing in real, concrete ways: LangChain shows you can match frontier model performance by tuning scaffolding instead of the model, Lenny builds a Claude harness for Sentry bug triage and walks through exactly how it works, and LangChain + NVIDIA ship a governed blueprint for running coding agents on sensitive enterprise workloads.

LangChain - Tuning the Harness, Not the Model#

Source: https://www.langchain.com/blog/tuning-the-harness-not-the-model-a-nemotron-3-ultra-playbook Credibility: High (first-party LangChain blog post with specific cost and performance claims)

What happened: LangChain published a detailed playbook showing they matched Anthropic Opus 4.8's best agent run using Nemotron 3 Ultra at roughly 8x lower cost. The key insight is that they only changed the scaffolding around the model, not the model itself. This is a direct counterargument to the default assumption that better performance requires a better (and more expensive) model.

Key technical details:

  • Performance matched to Opus 4.8's best agent run without any model fine-tuning
  • Cost reduction: approximately 8x lower vs. Opus 4.8
  • What they tuned: the harness — meaning prompts, tool call structure, retry logic, memory management, and control flow
  • The approach is replicable for any team running an open or open-weight model

Why it matters for PMs: This is one of the most directly useful signals for anyone making model selection and cost decisions right now. The default mental model is "better output = bigger model = higher cost." This post challenges that. If a significant portion of model performance comes from how you structure the agent loop rather than which model you pick, then model selection is only half the decision. Scaffolding quality and harness design become first-class product decisions, not engineering implementation details. That has real implications for build vs. buy, vendor selection, and how you staff and prioritize agent work.

Critical questions:

  • What tasks did they use to benchmark this? Agent performance varies enormously by domain and task type.
  • Is this reproducible at the task variety and volume a real product needs, or does it break down at edge cases?
  • Nemotron 3 Ultra is an open-weight model — what's the total cost when you factor in hosting, ops, and iteration time vs. a managed API?
  • How much engineering expertise is required to tune a harness well? If it takes a specialist, the 8x cost savings may not survive the org math.

Action you could take today: Pull up the cost breakdown for your highest-volume AI feature or agent workflow. Calculate what 8x cost reduction would mean in dollar terms over 12 months. That number is the business case for investing engineering time in harness optimization — if the number is big enough, this is worth a spike.

Lenny Rachitsky - Building a Claude Agent SDK Harness for Bug Triage#

Source: https://www.lennysnewsletter.com/p/what-a-harness-is-and-how-to-build Credibility: High (first-person account from a PM who built and shipped this, with concrete implementation details)

What happened: Lenny published a post and episode walking through a Claude Agent SDK harness he built to automate Sentry bug triage. He explains what a harness is, why it matters, and shows exactly how to build one — framed specifically for PMs, not engineers. This is a companion piece to the broader harness-tuning conversation above, but from the practitioner PM perspective.

Key capabilities:

  • Automates the "dear agent, please fix this bug" workflow using Sentry data
  • Built on Claude Agent SDK with a custom harness (structured prompts, context management, tool calls)
  • Lenny frames a harness as the "scaffolding that controls what the agent sees, does, and decides" — not the model itself
  • Designed to be owned and iterated on by a PM, not handed off to engineering

Why it matters for PMs: Two things are happening here. First, this is a concrete example of a PM doing what's increasingly expected: owning an AI workflow end-to-end, including the scaffolding. Second, the Sentry bug triage use case is a real, high-frequency PM pain point — the volume of bugs, the prioritization decisions, the context-switching. If Lenny's harness works as described, it's a template that's directly portable to any product team running a similar triage workflow. The framing matters too: "harness" is becoming the vocabulary for this layer of AI system design, and PMs who understand it will be able to have more useful conversations with their engineering partners.

Critical questions:

  • How much time did it take to build and iterate on the harness vs. time saved in triage? Net ROI matters.
  • Does the harness surface false positives that waste engineering time, or does it genuinely improve signal quality?
  • How does this scale if the Sentry volume is 10x higher? Does the harness hold up or does it need significant rework?
  • What happens when Claude makes a wrong call on a bug's severity? What's the error recovery path?

Action you could take today: If your team uses Sentry (or Linear, Jira, GitHub Issues), sketch out what a bug triage harness would need to know to make good decisions. List the inputs it needs (severity signals, affected user count, recency, component owner) and the outputs it should produce (priority tier, suggested assignee, brief context summary). That's the spec for a harness — and you can share it with an engineer as a starting point.

LangChain + NVIDIA - NemoClaw Deep Agents Blueprint for Enterprise Code#

Source: https://www.langchain.com/blog/deep-agents-code-on-nemoclaw-a-governed-blueprint-for-your-most-sensitive-code Credibility: High (first-party announcement from LangChain, with specific architecture and governance details)

What happened: LangChain and NVIDIA jointly launched NemoClaw, a blueprint for running Deep Agents Code on sensitive enterprise codebases. The architecture is built around security-first defaults: deny-by-default networking, human approval gates, and audit logs. It combines LangChain's Deep Agents Code with NVIDIA's Nemotron 3 Ultra and OpenShell, and is designed specifically for organizations that need coding agents but can't put sensitive code in a cloud API.

Key technical details:

  • Deny-by-default networking: agents can't call out to the internet unless explicitly permitted
  • Human approval gates built into the agent loop for high-stakes code changes
  • Audit logs for every agent action — who authorized what, when, what the output was
  • Runs on-premises or in a governed private cloud using NVIDIA infrastructure
  • Code modernization is the stated use case (legacy code migration, refactoring)

Why it matters for PMs: This is the enterprise unlock for coding agents. The blockers for large organizations adopting tools like Cursor or GitHub Copilot at scale are almost always: "we can't send our code to OpenAI's API," and "we need to know what the agent did and why." NemoClaw directly addresses both. If you're a PM at a financial institution, healthcare company, or any regulated enterprise, this is the architecture pattern your security and compliance teams have been waiting for. The human approval gate design is also worth studying as a pattern — it's a concrete answer to the "how much autonomy are we comfortable giving agents" question that every product team is wrestling with right now.

Critical questions:

  • Audit logs are listed as a feature, but what's the actual fidelity? Can you reconstruct exactly what context the agent had when it made a decision?
  • Human approval gates are only useful if humans can actually evaluate the proposed changes — how does the UX handle that at scale?
  • What's the total infrastructure cost of running this vs. a managed API, including GPU costs, ops overhead, and security review?
  • "Sensitive code modernization" is a high-stakes use case — what's the rollback path if an agent rewrites something incorrectly?

Action you could take today: If your company has a backlog of legacy code migration work, pull up the conversation about why it hasn't moved faster. Odds are the blockers are capacity and risk tolerance, not technical knowledge. This blueprint is a concrete thing to bring to that conversation — not as "let's do this tomorrow" but as "here's what the architecture for doing this safely looks like."

Quick Hits#

The Thread#

Agent scaffolding is becoming a product discipline, not just an engineering concern. Three items this week point the same direction: LangChain shows harness tuning can substitute for model upgrades, Lenny demonstrates a PM building and owning a harness end-to-end, and NemoClaw ships a governed blueprint with approval gates and audit logs baked in. The pattern is consistent: the agent loop architecture, not the model, is where product decisions are increasingly happening.

Sit With This#

LangChain tuned only the scaffolding around Nemotron 3 Ultra and matched Opus 4.8's performance at 8x lower cost. No model changes. Just better harness design.

For your product: Where are you currently treating model selection as the primary performance lever? Is there a workflow or agent loop on your roadmap where you've assumed "better output requires a better model" — and would it be worth running a harness-tuning experiment first?