Engineering at HeyGen: Inside the team building AI video at scale
How a Video Company Built the #1 Voice Model
Oct 26, 2026HeyGen let users choose between leading voice engines. By using real-world feedback and blind preference testing, HeyGen built a voice model that surpassed competitors in quality while reducing costs by millions.
HeyGen Voice: Scene-Aware Voice Cloning for Talking Videos
Oct 14, 2026An open beach. A moving car. A busy factory floor. HeyGen Voice brings your voice into the picture, shaping the delivery and sound to fit the scene.
Benchmarking Code-to-Video: Evaluating AI Agents on the Art of Motion Design
Sep 18, 2026
Video Podcast: Enforcing Continuity Across Independently Rendered Turns
Aug 24, 2026Each speaker’s stretch of dialogue in a Video Podcast is rendered independently. The challenge is making those clips behave as though they came from the same recording.
HeyGen x Google Cloud: Bringing Avatar IV to TPUs
Aug 13, 2026HeyGen is an AI video generation platform with industry-leading avatar models. Avatar IV, the diffusion stack behind our talking-head videos, runs on more than 18B parameters and is available via Web and API. Working with the Google Cloud AI-Ninja team, we brought Avatar IV to an eight-chip Trillium (v6e) host and made it 1.86× faster than our first working version. It now matches the performance of our 8×H100 production setup while being up to 25% more cost-efficient per minute of generated video.
How Temporal Powers Workflows at HeyGen
Aug 5, 2026How we moved from a custom job system to durable execution across a multi-cloud GPU fleet.
Reframe Context Engineering - Lessons from Building Video Agent(s) - Part 1
Jul 27, 2026Context engineering is very different when your agent's goal is multi-modal (e.g. videos). Here's what we learned shipping one.
One Model, Multiple Controllability: Composable Behavior LoRAs for Video Avatars
Jul 23, 2026The expensive part of multi-behavior avatar control is a dataset where every behavior co-occurs, labeled, at studio quality — it barely exists. We skip it: train each behavior as a separate LoRA on cheap single-attribute data, compose them at inference, and close the gap with a short fine-tune on only a little real overlap.
Driving Real-Time Actions from Avatar Speech: A Word-Clock Architecture for the HeyGen Realtime API
Jul 15, 2026We synchronize AI avatar speech with live browser actions using per-word timestamps emitted by the Avatar Realtime API. By anchoring clicks, highlights, and scrolling to the exact word being spoken, the system keeps actions aligned with natural speech instead of unreliable timers. The approach powers interactive AI agents that can browse the web, teach, review code, and respond live while maintaining accurate audiovisual synchronization.
Auto-Optim: A Blind Agent Matched Our Hard-Won 3x, With No human in the Loop
Jun 29, 2026We handed Claude Code a benchmark, a tamper-proof quality gate, and one rule file, then walked away. On a single H100 it made the Wan 2.2 video decoder 3× faster on its own. The same speedup cost us 105 hand-steered experiments the first time; this run took 18, blind.
