Home
Jul 21, 2026
View All

How PMs Actually Use AI Coding Tools (And Why Replit's Numbers Are Wild)

·2 underrepresented voices

The Short Version#

Replit engineers tripled code output in six months using AI agents; Lenny's Morning Brew interview reveals how Claude workflows actually work at scale (interviews + voice-coded markdown + multi-persona revision loops); and Microsoft is betting hard on local inference by partnering with Mistral for enterprise adoption — the pattern is shifting from "AI helps you code" to "AI handles entire workflows and runs everywhere."

Amjad Masad (Replit) — Engineers Nearly Tripled Code Output in Six Months#

Source: https://x.com/amasad/status/2077802290304684404 Credibility: High (founder's direct post about production usage)

What happened: Amjad Masad, Replit CEO, posted that engineers at Replit have nearly tripled code output in the past six months using AI agents. This is a direct signal about what happens when developers adopt agentic workflows at scale — not a benchmark claim, not a controlled study, but an actual company shipping and observing real developer behavior change.

Key technical details:

  • ~3x code output increase observed across engineering teams
  • Timeframe: last six months (roughly Dec 2025–Jun 2026)
  • Context: Replit's platform already embeds AI agents heavily in its product, so this reflects developers who are comfortable with AI-assisted workflows
  • This is production usage across a team, not a single developer experiment

Why it matters for PMs: This is the clearest signal yet that agentic AI doesn't just improve velocity incrementally — it can fundamentally change what a developer outputs in a sprint. If your PM team is still thinking about AI as "a tool that helps developers write faster," you're underestimating the shift. The 3x number matters because it suggests a category shift: individual developers using AI to write code faster vs. teams running AI agents that handle entire features or workflows differently. The implications for sprint planning, QA, and code review are massive.

Critical questions:

  • What's the quality impact? Is the 3x output pure lines of code, or is it functional features? Code that ships vs. code that requires rework?
  • Which types of work saw the biggest multiplier — greenfield features, refactoring, or test coverage?
  • What's the onboarding/adoption curve? Did all engineers reach 3x, or is this an average across some reaching 5x and others at 1x?
  • Are there classes of work where AI agents reduce velocity (security-sensitive code, system-critical paths)?

Action you could take today: If you're building an AI coding tool, ask your most active users for their output metrics over the last two quarters. Don't ask "how much faster are you?" — ask for actual commit frequency, PRs per week, or story points completed. Replit's 3x is worth benchmarking against.

Alex Lieberman (Morning Brew) — How a Claude Workflow Actually Works at Scale#

Source: https://www.lennysnewsletter.com/p/how-the-founder-of-morning-brew-built Credibility: High (detailed case study from Lenny Rachitsky's interview series)

What happened: Lenny interviewed Alex Lieberman (Morning Brew founder) about his Claude-powered content machine. The workflow is concrete and illuminating: Claude interviews Alex before drafting, encodes his voice in Markdown, runs six-persona revision loops before posting — and never outputs what Alex describes as "slop." This is a rare look at how someone actually scales content creation with AI without it becoming generic.

Key technical details:

  • Pre-draft interview loop: Claude asks questions to capture Alex's perspective and tone before writing
  • Voice encoding: His editorial voice is coded into the prompt/system instructions using Markdown-formatted examples and rules
  • Revision cycle: Six distinct personas (editor, fact-checker, brand voice, clarity reviewer, engagement optimizer, final polish) critique and refine the draft sequentially
  • Output quality bar: Alex explicitly rejects output that sounds like "AI-generated content"
  • Frequency: Suggests this workflow runs regularly enough to power Morning Brew's content schedule

Why it matters for PMs: This is a pattern that works and scales. The insight isn't "Claude is good at writing" — it's that multi-turn, structured workflows with persona-based review produce content that humans actually want to read. If you're building AI features into content tools, email platforms, or creator platforms, this workflow is the template: (1) gather context through dialogue, (2) encode user voice as system-level instructions, (3) route drafts through specialized personas, (4) maintain a quality gate. The "slop" problem isn't solved by better models — it's solved by structured workflows that enforce voice consistency and editorial standards.

Critical questions:

  • How much of the work is the workflow design vs. the Claude model quality? Would this still work with GPT-4 or Gemini?
  • What's the time investment? Does the six-persona loop require manual intervention, or does Claude handle each persona autonomously?
  • How does this scale to other content types beyond Morning Brew's format (newsletters, long-form, social, video scripts)?
  • What's the failure rate? How often does output require more than the six-persona revision loop?

Action you could take today: If you're shipping AI-assisted writing, sketch out your revision loop. How many passes does content go through? Who (or what) reviews it, and in what order? Try mapping Alex's workflow to your product — you'll quickly see gaps in your review architecture.

Microsoft / Mistral Strategic Partnership — Local Models for Enterprise#

Source: https://news.microsoft.com/source/2026/07/21/microsoft-and-mistral-expand-strategic-partnership-to-give-enterprises-and-regulated-industries-frontier-ai-they-can-control/ Credibility: High (official announcement)

What happened: Microsoft announced an expansion of its strategic partnership with Mistral AI, specifically focused on bringing local and hybrid inference capabilities to enterprises and regulated industries. The framing is explicit: enterprises want AI they can "control." This is a direct response to the tension between capability (use frontier models) and compliance (keep data on-prem or air-gapped). Mistral becomes Microsoft's non-OpenAI answer for customers who can't use cloud-hosted models.

Key technical details:

  • Focus on enterprise and regulated industries (finance, healthcare, government)
  • Local and hybrid inference as the primary value prop
  • Builds on existing partnership (Mistral models available on Azure)
  • Emphasis on "frontier AI" — still state-of-the-art, just with deployment options
  • Likely includes integration with Azure infrastructure and security tooling

Why it matters for PMs: This is a bet that the future of enterprise AI isn't just better models — it's better control over where models run. If you're selling to regulated industries or enterprises with strict data residency requirements, you're watching this deal closely. Microsoft is essentially saying: "You can't use OpenAI because of data sovereignty — here's Mistral." The pattern validates what we've been seeing all year: enterprises will tolerate slightly less capable models if it means keeping inference on-prem or in trusted regions. For product teams building on top of APIs, this means your enterprise buyers may care less about "latest model" and more about "can this run in our VPC?"

Critical questions:

  • What's the performance/capability gap between Mistral's models and GPT-5/Claude when run on-prem vs. cloud? Do customers actually notice?
  • Which regulated verticals are the beachhead? Finance first, then healthcare and gov?
  • How deep is the Azure integration? Is this just "Mistral models available in Azure," or is there platform-level optimization?
  • Does this cannibalize Azure OpenAI? Are customers choosing between "use OpenAI with controls" vs. "use Mistral on-prem"?

Action you could take today: If you sell to enterprise or regulated industries, ask your customers about their data residency requirements for AI. Find out which are currently blocked from using cloud-hosted models. Mistral partnership signals that this constraint is real enough for Microsoft to make a public bet — it's worth understanding if it affects your product roadmap.

Quick Hits#

The Thread#

From code output to workflow control. Three separate signals today point to the same shift: Replit's 3x code output (agents do more than assist), Morning Brew's Claude workflow (humans define control, AI executes within constraints), and Microsoft-Mistral partnership (enterprises want AI on their terms). The common thread isn't "AI is getting better" — it's "who decides how AI works, and where." Replit's engineers are ceding control to agents in greenfield work. Alex is maintaining tight editorial voice through structured prompts. Enterprises want models they can run anywhere. Different constraints, same question: how much autonomy, and who sets the boundaries?

Sit With This#

Replit engineers are tripling code output with AI agents, but Amjad didn't share the quality metrics — just the velocity multiplier. Morning Brew's Alex Lieberman built an entire revision pipeline (six personas) to make sure Claude output doesn't sound like "slop," suggesting he doesn't trust the model alone.

For your AI product: If your users are seeing massive velocity gains but you haven't measured quality impact yet, what happens when they do? How would you know if faster code is creating more bugs, security issues, or technical debt that surfaces three sprints later?