How AGENTS.md Outperforms Skills & Clawdbot Takes Over PCs

Vercel - AGENTS.md Outperforms Skills in Agent Evaluations#

Source: https://vercel.com/blog/agents-md-outperforms-skills-in-our-agent-evals Credibility: High (controlled evaluation with documented methodology)

What happened: Vercel ran evaluations comparing AGENTS.md (plain documentation) versus packaged skills for agent capabilities. AGENTS.md consistently outperformed skills across their eval suite.

Key findings from the benchmark:

The comparison tested two approaches:

1. Skills: Packaged bash tools following the agentskills.io spec
2. AGENTS.md: Plain markdown documentation describing available capabilities

Why AGENTS.md won:

• Lower abstraction overhead: Models directly see what's available without parsing package metadata
• Better context for planning: Documentation format provides richer context about when and how to use capabilities
• Fewer failure modes: No package installation, versioning, or dependency conflicts
• Model-native format: LLMs are trained on documentation, not package managers

The architectural insight: Vercel isn't abandoning skills—they're hybrid. Skills remain valuable for cross-platform distribution and version management. But for single-agent contexts where the agent has direct filesystem access, documentation beats packages.

Why it matters for PMs: This challenges the "npm for agents" mental model. If plain documentation outperforms packaged skills in evals, the question shifts: when is the packaging overhead worth it? The answer appears to be: when you need cross-platform distribution or version control. For single-agent systems, simpler is better.

Critical questions:

• What specific eval tasks did they test? (Not disclosed—methodology matters)
• Does this generalize beyond their specific agent architecture?
• At what scale does skill packaging become necessary despite eval performance?
• How much of the performance gap is model-specific (Claude vs. GPT-4 vs. others)?

Action you could take today: If you're building an agent that controls one environment, try plain AGENTS.md documentation before adding skill packages. Document your capabilities in markdown, test if the model can discover and use them, and only add packaging if discovery fails.

Lenny Rachitsky - Clawdbot: Autonomous AI with Full PC Control#

Source: https://www.lennysnewsletter.com/p/i-gave-clawdbot-now-moltbot-access Credibility: High (detailed first-person experiment with specific examples)

What happened: Lenny gave Clawdbot (now Moltbot, from Anthropic's Cowork) full access to his computer—calendar, email, browser, and files. He documented what it can do autonomously and where it currently breaks.

What Clawdbot actually did:

Successful autonomous tasks:

• Calendar management: Automatically scheduled meetings by checking availability and sending invites
• Email triage: Sorted inbox, drafted responses for approval, flagged urgent items
• Web research: Searched for information across multiple sites, synthesized findings
• File organization: Reorganized project folders based on Lenny's instructions
• Multi-step workflows: Combined tasks (e.g., research topic → draft email → schedule follow-up)

Where it struggled:

• Authentication boundaries: Hit login walls on some sites despite browser access
• Ambiguous instructions: Required clarification when goals weren't specific
• Error recovery: Sometimes got stuck when a task failed partway through
• Context limits: Lost track of multi-day projects without explicit state management

The key pattern—"chief of staff" model: Clawdbot works best as a delegated executor: you describe what you want done, it figures out how. This matches the LangSmith Agent Builder framing from Jan 26: agents as chief of staff, not just code runners.

Why it matters for PMs: This is the first mainstream PM demonstrating full PC control to an audience of thousands of product managers. It normalizes autonomous agents managing real work—not demos. For PMs evaluating agent capabilities, the message is: PC control works for high-volume, low-ambiguity tasks (email triage, scheduling). It doesn't yet work for high-stakes decisions or ambiguous goals.

Critical questions:

• What's the error rate on autonomous actions? (Not disclosed—we need failure data)
• How much supervision did Lenny provide versus true autonomy?
• What happens when the agent makes a mistake in a high-stakes workflow (e.g., deletes wrong file)?
• Security model—how does Anthropic ensure agents don't exfiltrate sensitive data?

Action you could take today: Identify one repetitive PM task that fits the "chief of staff" pattern—meeting prep, inbox management, research compilation. Try delegating it to an agent with PC access (Clawdbot, Cursor, or similar). Document where it succeeds versus where you had to intervene. This calibrates realistic expectations.

Microsoft - Work IQ: Organizational Context via MCP#

Source: https://www.linkedin.com/posts/satyanadella_so-much-of-dev-work-happens-in-the-context-activity-7420485585376620544-vudJ Credibility: High (first-party announcement, MCP server available now)

What happened: Microsoft launched Work IQ, an MCP server connecting organizational data (emails, meetings, documents, Teams, SharePoint) to GitHub Copilot CLI. Developers can now query institutional knowledge directly from the terminal.

Key technical details:

• Five data categories: Emails, calendar, documents, Teams messages, people directory
• CLI tool: npm install -g @microsoft/workiq
• MCP integration: Uses Model Context Protocol (same as Anthropic's Claude)
• Example use case: Compare specs in SharePoint against actual code to find requirement drift

The MCP adoption signal: Microsoft adopting Anthropic's MCP protocol is notable—it validates MCP as the de facto standard for context integration, not a proprietary Anthropic feature. For PMs building AI tools that need organizational context, this suggests MCP is the integration layer to build against.

Why it matters for PMs: This addresses the "context gap" problem: AI tools lack access to institutional knowledge, so they give generic answers. Work IQ solves this by connecting Copilot to the same data sources humans use (emails, docs, meetings). For PMs building enterprise AI features, the pattern is: integrate organizational context, don't just improve the model.

Critical questions:

• Licensing: Requires both GitHub Copilot AND Microsoft 365 Copilot—what's the real adoption barrier?
• Security: Security researchers flag prompt injection risks when agents access emails/documents—how does Microsoft mitigate exfiltration?
• Enterprise concerns: 67% of security teams worry about AI exposing sensitive data—how does this affect rollout?

Action you could take today: If you're building AI features that need organizational context, evaluate MCP as your integration protocol. Microsoft's adoption validates it as enterprise-ready. Even if you're not building for GitHub Copilot, MCP is becoming the standard connector layer.