LangSmith's Memory System & Video Generation on AI Gateway

LangChain - How to Use Memory in Agent Builder#

Source: https://blog.langchain.com/how-to-use-memory-in-agent-builder/ Credibility: High (detailed product guide with concrete implementation patterns)

What happened: LangChain launched memory capabilities in Agent Builder—enabling agents to remember user corrections, preferences, and successful approaches across sessions. The key shift: agents now improve through usage, not just through prompt engineering or retraining.

Key memory patterns:

What gets remembered:

• Corrections: When users fix agent outputs, those corrections apply to future similar tasks
• Preferences: User feedback on tone, format, or approach persists across sessions
• Successful patterns: Approaches that worked well get prioritized in future executions
• Context about user: Domain knowledge, typical workflows, and recurring tasks

How memory works architecturally:

• Stored as structured context, not raw conversation logs
• Applied automatically when relevant to new tasks
• Users can view, edit, or delete stored memory
• Memory scoped per agent (not shared across different agents)

The feedback loop pattern: Instead of teaching agents through prompt engineering (static instructions), memory enables teaching through usage:

1. Agent produces output
2. User corrects or provides feedback
3. Agent stores that feedback as memory
4. Next time similar task appears, agent applies learned pattern

Example workflow:

• First time: Agent generates SQL query, user corrects column name
• Second time: Agent remembers correct column name, generates accurate query
• Over time: Agent learns user's preferred query patterns, naming conventions, and common joins

Why this matters vs. conversational memory: Conversational memory (ChatGPT remembering earlier in the session) is short-term. LangSmith's memory is long-term and cross-session—the agent gets smarter with repeated use, not just within one conversation.

Why it matters for PMs: This continues the shift from Jan 29's context management patterns and Feb 9's agentic memory systems. For PMs building agents, memory changes the product model: agents become assets that improve over time, not stateless utilities. The question becomes: how do you design feedback loops that actually teach agents, versus just creating noise?

Critical questions:

• What's the signal-to-noise ratio on user feedback? Do corrections teach useful patterns or just clutter memory?
• How do you prevent memory from encoding user mistakes or bad practices?
• Privacy model: where is memory stored, who can access it, can it be exported?
• When does accumulated memory create technical debt—outdated patterns that should be unlearned?
• How do you validate that memory actually improves agent performance versus creating brittleness?

Action you could take today: If you're building agents that users interact with repeatedly, design one feedback loop explicitly: what should the agent remember from this interaction? Implement simple memory storage (even just a JSON file) for one workflow. Test whether remembering past corrections actually improves future outputs or just adds complexity.

Vercel - Video Generation with AI Gateway#

Source: https://vercel.com/blog/video-generation-with-ai-gateway Credibility: High (first-party infrastructure launch with technical details)

What happened: Vercel launched video generation capabilities through AI Gateway—adding Grok Imagine Video, Wan models, Kling, and Veo to their unified API. The pattern: multimodal infrastructure expanding beyond text and images to video generation.

Key video generation models added:

Grok Imagine Video:

• Text-to-video generation
• Multiple aspect ratios supported
• Streaming generation (progressive rendering)

Wan models:

• Chinese video generation models
• Specialized for different video styles
• Cost-optimized for Asian market use cases

Kling:

• High-quality video synthesis
• Longer video duration support
• Advanced motion control

Veo (Google's video model):

• Experimental video generation
• Google Cloud ecosystem integration
• Early access through Vercel Gateway

Why Gateway matters for video: Video generation models have different:

• Pricing: Per-second video costs vary wildly by provider
• Latency: Some models take minutes; others seconds
• Quality: Motion coherence, style consistency differ significantly
• Capabilities: Duration limits, resolution, aspect ratios vary

AI Gateway provides unified API, automatic failover, and cost tracking across providers—solving the same orchestration problems for video that it solved for text/image generation.

The infrastructure insight: Video generation isn't just "bigger prompts"—it requires different infrastructure:

• Longer processing times (async workflows, not real-time)
• Larger outputs (storage and bandwidth considerations)
• Preview workflows (progressive rendering vs. final output)
• Cost management (video generation is expensive; need routing and caching)

Why it matters for PMs: This validates multimodal infrastructure as table stakes for AI products. For PMs building features that might need video (product demos, content creation, visualization), the question shifts from "should we support video?" to "which video models through which infrastructure?" Gateway abstracts provider complexity, but PMs still need to understand cost-quality tradeoffs per model.

Critical questions:

• What's the cost per video across different providers? (No public pricing in announcement)
• Latency distribution: how long does video generation typically take?
• Quality comparison: which models work best for which use cases (product demos vs. artistic content)?
• Storage implications: where do generated videos live, and who pays for bandwidth?
• Does Gateway cache video outputs, or regenerate on each request?

Action you could take today: If you're considering video generation features, prototype one workflow through Vercel's AI Gateway. Generate a 5-second video across different models (Grok, Kling, Veo) with the same prompt. Compare: cost, latency, quality, and whether Gateway's unified API actually simplifies orchestration versus calling models directly.

Teresa Torres - Building AI Sales Reps: How ShowMe Orchestrates Voice, Video, and Screen Sharing#

Source: https://www.producttalk.org/building-ai-sales-reps-showme/ Credibility: High (detailed case study with technical architecture and product decisions)

What happened: Teresa published a case study on ShowMe—a company building AI sales representatives that handle voice calls, video interactions, and screen sharing for product demos. The architecture reveals how multimodal agents orchestrate real-time interactions beyond text.

Key orchestration patterns:

The multimodal workflow:

1. Voice input: Agent transcribes and processes spoken questions in real-time
2. Screen sharing: Agent sees prospect's screen, understands context visually
3. Video analysis: Agent reads facial cues and engagement signals
4. Voice output: Agent responds conversationally with synthesized speech
5. Screen control: Agent demonstrates features by controlling shared screen

Why this is hard technically:

• Latency constraints: Voice conversations require <500ms response time (text can tolerate 2-3 seconds)
• Context switching: Agent must process voice, visual, and screen context simultaneously
• Tool coordination: Screen control, video analysis, and voice synthesis must synchronize
• Error recovery: Voice errors are harder to recover from than text (can't just regenerate)

The architectural decision: ShowMe uses subagent pattern (from Jan 28 Deep Agents, Feb 5 Deep Research):

• Conversation agent: Handles voice interaction, maintains dialogue flow
• Visual agent: Processes screen sharing and video, extracts relevant context
• Demo agent: Controls product demo, navigates UI based on conversation
• Coordinator: Orchestrates subagents, ensures synchronized responses

Product insights from the case study:

What works:

• Prospects engage longer with AI sales reps than with static demos (8 minutes vs. 3 minutes average)
• Screen sharing increases conversion (prospects see product in their context, not generic demo)
• Voice interaction feels more natural than chat for product questions

What doesn't work (yet):

• Complex questions require human handoff (AI can't handle objections or negotiate pricing)
• Video analysis is weak signal (facial cues don't reliably predict buying intent)
• Error recovery breaks experience (when agent misunderstands, restarting conversation loses context)

Why it matters for PMs: This demonstrates multimodal agent orchestration for a real product use case—not research or prototypes. For PMs building agents that need to operate across voice, video, and screen sharing, ShowMe's architecture provides reference patterns. The key insight: multimodal agents require coordination layers, not just bigger models.

Critical questions:

• What's the cost per demo session? (Voice, video, and screen processing at scale)
• Human handoff rate: what percentage of conversations escalate to human sales?
• Conversion lift: do AI sales reps actually increase deals closed, or just engagement time?
• Privacy: how does ShowMe handle screen sharing data (sensitive user information)?
• When does orchestration complexity outweigh the value of multimodal interaction?

Action you could take today: If you're building agents that need voice or video capabilities, map your orchestration requirements: Which modalities need real-time coordination? What's your latency budget? Where do you need human handoff? This clarifies whether you need subagent patterns or if single-modal (text-only) agents suffice.