HeyGen HyperFrames: The HTML-First Video Framework Built for AI Agents

The Core Idea

HeyGen open-sourced HyperFrames in 2026 under Apache 2.0. The tagline is three sentences: "Write HTML. Render video. Built for agents."

Instead of defining videos with React components (Remotion) or timeline APIs (traditional editors), you write plain HTML with data-* attributes. GSAP handles motion. AI coding agents like Claude Code, Cursor, Codex, and Gemini CLI are the primary intended users.

The homepage headline makes the intent explicit: "Now Claude Code can edit videos."

Why HTML Instead of React

The design decision becomes clear once you consider what AI agents are actually good at. They read and modify HTML fluently. React introduces a translation layer — the agent has to reason about component trees, state, and build pipelines before a single pixel renders.

HyperFrames removes that layer:

<div id="root" data-composition-id="root"
     data-start="0" data-width="1920" data-height="1080">
  <video id="clip-1" data-start="0" data-duration="5" data-track-index="0"
         src="intro.mp4" muted playsinline></video>
  <img id="overlay" class="clip" data-start="2" data-duration="3"
       data-track-index="1" src="logo.png" />
  <audio id="bg-music" data-start="0" data-duration="9" data-track-index="2"
         data-volume="0.5" src="music.wav"></audio>
</div>

You have four clip types (<video>, <img>, <audio>, nested <div data-composition-id>) and five attributes (data-start, data-duration, data-track-index, data-volume, class="clip"). That is the full authoring surface.

One-Line Install

npx skills add heygen-com/hyperframes

This command installs five slash commands into your agent environment:

• /hyperframes — Composition authoring
• /hyperframes-cli — init, preview, render, lint, transcribe, tts
• /hyperframes-registry — Install blocks and components
• /website-to-hyperframes — URL → video pipeline
• /gsap — GSAP timeline and easing reference

After installation, the agent already understands the framework's conventions and structure.

Deterministic Rendering as Non-Negotiable

One of the seven official "must follow" rules states:

Math.random() is forbidden. If you need randomness, use a seeded PRNG like mulberry32.

That level of commitment to determinism is unusual in video tooling. The rationale is clear: automated pipelines require identical output for identical input. Otherwise you cannot integrate rendering into CI/CD.

Other non-negotiable rules:

1. All timelines must register into window.__timelines
2. <video> elements must be muted; audio goes into separate <audio> tags
3. GSAP timeline construction must be synchronous (no async/await/fetch)
4. Timed elements require class="clip"
5. Scripts must not call video.play() or audio.currentTime — the framework owns media control
6. Every scene needs an entrance animation
7. Every scene transition needs an explicit declaration

Natural Language to Technical Mapping

The prompting guide's translation tables do heavy lifting.

Motion & Easing

Caption Tones

Audio-Reactive Mapping

Recommended intensity: text 3-6%, background 10-30%.

Two Prompting Modes

Cold Start

Describe the video from scratch.

10-second product intro, fade-in title, dark background, BGM, corporate mood

Recommended structure: Duration, Aspect ratio, Mood, Key elements.

Warm Start

Feed existing context and let the agent compose the video.

Turn this GitHub repo into a 45-second pitch video
Convert this PDF into a 30-second summary video

This is where HyperFrames genuinely changes the workflow. The /website-to-hyperframes slash command handles URL → video as a first-class pipeline, letting the agent handle both research and production in a single prompt.

Common Mistakes Worth Knowing

The Common Mistakes documentation is a debugging checklist. Highlights:

1. Animating video element dimensions freezes rendering.

// ❌ Browser stops rendering frames
gsap.to('#video1', { width: 1920, duration: 1 });

// ✅ Animate a wrapper div instead
gsap.to('.video-wrapper', { width: 1920, duration: 1 });

2. Timeline length less than video length.

// Extend timeline with a zero-duration set
tl.set({}, {}, 283);

3. Oversized images cause massive decode cost.

A 7000×5000 PNG causes a ~140MB decode every frame. Keep images at 2× canvas size maximum.

4. Backdrop-filter stacks tank performance.

Calculating 16 layers of backdrop-filter: blur() every frame destroys render time. Cap at 2-3 layers.

Architecture

The monorepo separates responsibilities cleanly:

The Frame Adapter pattern is the extensibility point. Adapters bring GSAP, Lottie, CSS, Three.js, and other animation runtimes into the render pipeline. The first community-contributed adapters will probably shape the ecosystem direction.

TTS Built-In

Kokoro runs locally. No API key required.

npx hyperframes tts --text "Hello world" --voice af_heart --output narration.wav

Voice recommendations:

• Product demos: af_heart, af_nova
• Tutorials: am_adam, bf_emma
• Marketing: af_sky, am_michael

Component Registry

Over 50 installable blocks:

npx hyperframes add flash-through-white
npx hyperframes add instagram-follow
npx hyperframes add data-chart

Categories span social overlays, shader transitions, data visualizations, and cinematic effects.

Recommended Workflow

1. npx hyperframes init my-video (installs the skill automatically)
2. Open the project in Claude Code, Cursor, Codex, or Gemini CLI
3. Use /hyperframes with a warm start prompt
4. npx hyperframes preview for browser live reload
5. Iterate with small, targeted prompts: "make the title 2x larger", "add a fade-out at the end"
6. npx hyperframes lint to catch structural issues
7. npx hyperframes render --preset high --output final.mp4

Anti-Patterns

The prompting guide explicitly calls out three:

• Requesting React or Vue components — adds a translation layer
• Requesting 4K at 60fps — 1920×1080 at 30fps is faster and usually sufficient
• Skipping the slash command — the agent will fall back to generic HTML video conventions

System Requirements

Node.js 22 or higher and FFmpeg. That is the entire list.

Why This Matters

HyperFrames signals a specific architectural bet: content tooling is moving toward agent-primary, human-secondary design. Most frameworks retrofit agent support; HyperFrames treats it as the foundational constraint.

Whether or not this particular framework becomes dominant, the design choices — HTML-first, deterministic rendering, slash command integration, thin core with a Frame Adapter extension pattern — are worth studying. They represent a coherent vision of what agent-native tooling looks like.

Links

• Homepage: https://hyperframes.heygen.com/
• Prompting guide: https://hyperframes.heygen.com/guides/prompting
• Compositions concept: https://hyperframes.heygen.com/concepts/compositions
• Common mistakes: https://hyperframes.heygen.com/guides/common-mistakes
• GitHub: https://github.com/heygen-com/hyperframes