# HeyGen HyperFrames: The HTML-First Video Framework Built for AI Agents


## The Core Idea

HeyGen open-sourced **HyperFrames** in 2026 under Apache 2.0. The tagline is three sentences: **"Write HTML. Render video. Built for agents."**

Instead of defining videos with React components (Remotion) or timeline APIs (traditional editors), you write plain HTML with `data-*` attributes. GSAP handles motion. AI coding agents like Claude Code, Cursor, Codex, and Gemini CLI are the primary intended users.

The homepage headline makes the intent explicit: **"Now Claude Code can edit videos."**

## Why HTML Instead of React

The design decision becomes clear once you consider what AI agents are actually good at. They read and modify HTML fluently. React introduces a translation layer — the agent has to reason about component trees, state, and build pipelines before a single pixel renders.

HyperFrames removes that layer:

```html
<div id="root" data-composition-id="root"
     data-start="0" data-width="1920" data-height="1080">
  <video id="clip-1" data-start="0" data-duration="5" data-track-index="0"
         src="intro.mp4" muted playsinline></video>
  <img id="overlay" class="clip" data-start="2" data-duration="3"
       data-track-index="1" src="logo.png" />
  <audio id="bg-music" data-start="0" data-duration="9" data-track-index="2"
         data-volume="0.5" src="music.wav"></audio>
</div>
```

You have four clip types (`<video>`, `<img>`, `<audio>`, nested `<div data-composition-id>`) and five attributes (`data-start`, `data-duration`, `data-track-index`, `data-volume`, `class="clip"`). That is the full authoring surface.

## One-Line Install

```bash
npx skills add heygen-com/hyperframes
```

This command installs five slash commands into your agent environment:

- `/hyperframes` — Composition authoring
- `/hyperframes-cli` — init, preview, render, lint, transcribe, tts
- `/hyperframes-registry` — Install blocks and components
- `/website-to-hyperframes` — URL → video pipeline
- `/gsap` — GSAP timeline and easing reference

After installation, the agent already understands the framework's conventions and structure.

## Deterministic Rendering as Non-Negotiable

One of the seven official "must follow" rules states:

> **Math.random() is forbidden.** If you need randomness, use a seeded PRNG like mulberry32.

That level of commitment to determinism is unusual in video tooling. The rationale is clear: automated pipelines require identical output for identical input. Otherwise you cannot integrate rendering into CI/CD.

Other non-negotiable rules:

1. All timelines must register into `window.__timelines`
2. `<video>` elements must be `muted`; audio goes into separate `<audio>` tags
3. GSAP timeline construction must be synchronous (no `async/await/fetch`)
4. Timed elements require `class="clip"`
5. Scripts must not call `video.play()` or `audio.currentTime` — the framework owns media control
6. Every scene needs an entrance animation
7. Every scene transition needs an explicit declaration

## Natural Language to Technical Mapping

The prompting guide's translation tables do heavy lifting.

### Motion & Easing

| Natural Language | GSAP Easing | Use Case |
|-----------------|-------------|----------|
| smooth | `power2.out` | Natural deceleration |
| snappy | `power4.out` | Fast and decisive |
| bouncy | `back.out` | Overshoot and settle |
| springy | `elastic.out` | Oscillate and settle |
| dramatic | `expo.out` | Fast start, long glide |
| dreamy | `sine.inOut` | Slow and symmetric |

### Caption Tones

| Tone | Font | Animation | Size |
|------|------|-----------|------|
| Hype | Heavy weight | Scale-pop | 72-96px |
| Corporate | Clean sans-serif | Fade+Slide | 56-72px |
| Tutorial | Monospace | Typewriter | 48-64px |
| Storytelling | Serif | Slow fade | 44-56px |
| Social | Rounded playful | Bounce | 56-80px |

### Audio-Reactive Mapping

| Frequency/Property | Visual Property |
|-------------------|----------------|
| Bass | scale (beat pulse) |
| Treble | glow (shimmer) |
| Amplitude | opacity (breathing) |
| Mids | shape (morphing) |

Recommended intensity: text 3-6%, background 10-30%.

## Two Prompting Modes

### Cold Start

Describe the video from scratch.

```
10-second product intro, fade-in title, dark background, BGM, corporate mood
```

Recommended structure: Duration, Aspect ratio, Mood, Key elements.

### Warm Start

Feed existing context and let the agent compose the video.

```
Turn this GitHub repo into a 45-second pitch video
Convert this PDF into a 30-second summary video
```

This is where HyperFrames genuinely changes the workflow. The `/website-to-hyperframes` slash command handles URL → video as a first-class pipeline, letting the agent handle both research and production in a single prompt.

## Common Mistakes Worth Knowing

The Common Mistakes documentation is a debugging checklist. Highlights:

**1. Animating video element dimensions freezes rendering.**

```javascript
// ❌ Browser stops rendering frames
gsap.to('#video1', { width: 1920, duration: 1 });

// ✅ Animate a wrapper div instead
gsap.to('.video-wrapper', { width: 1920, duration: 1 });
```

**2. Timeline length less than video length.**

```javascript
// Extend timeline with a zero-duration set
tl.set({}, {}, 283);
```

**3. Oversized images cause massive decode cost.**

A 7000×5000 PNG causes a ~140MB decode every frame. Keep images at 2× canvas size maximum.

**4. Backdrop-filter stacks tank performance.**

Calculating 16 layers of `backdrop-filter: blur()` every frame destroys render time. Cap at 2-3 layers.

## Architecture

The monorepo separates responsibilities cleanly:

| Package | Role |
|---------|------|
| `hyperframes` | CLI (create, preview, lint, render) |
| `@hyperframes/core` | Types, parser, linter, runtime, frame adapter |
| `@hyperframes/engine` | Page → video capture (Puppeteer + FFmpeg) |
| `@hyperframes/producer` | Full pipeline (capture + encode + audio mix) |
| `@hyperframes/studio` | Browser-based composition editor |
| `@hyperframes/player` | Embeddable `<hyperframes-player>` web component |
| `@hyperframes/shader-transitions` | WebGL shader transitions |

The **Frame Adapter** pattern is the extensibility point. Adapters bring GSAP, Lottie, CSS, Three.js, and other animation runtimes into the render pipeline. The first community-contributed adapters will probably shape the ecosystem direction.

## TTS Built-In

Kokoro runs locally. No API key required.

```bash
npx hyperframes tts --text "Hello world" --voice af_heart --output narration.wav
```

Voice recommendations:
- Product demos: `af_heart`, `af_nova`
- Tutorials: `am_adam`, `bf_emma`
- Marketing: `af_sky`, `am_michael`

## Component Registry

Over 50 installable blocks:

```bash
npx hyperframes add flash-through-white
npx hyperframes add instagram-follow
npx hyperframes add data-chart
```

Categories span social overlays, shader transitions, data visualizations, and cinematic effects.

## Recommended Workflow

1. `npx hyperframes init my-video` (installs the skill automatically)
2. Open the project in Claude Code, Cursor, Codex, or Gemini CLI
3. Use `/hyperframes` with a warm start prompt
4. `npx hyperframes preview` for browser live reload
5. **Iterate with small, targeted prompts**: "make the title 2x larger", "add a fade-out at the end"
6. `npx hyperframes lint` to catch structural issues
7. `npx hyperframes render --preset high --output final.mp4`

## Anti-Patterns

The prompting guide explicitly calls out three:

- **Requesting React or Vue components** — adds a translation layer
- **Requesting 4K at 60fps** — 1920×1080 at 30fps is faster and usually sufficient
- **Skipping the slash command** — the agent will fall back to generic HTML video conventions

## System Requirements

Node.js 22 or higher and FFmpeg. That is the entire list.

## Why This Matters

HyperFrames signals a specific architectural bet: **content tooling is moving toward agent-primary, human-secondary design**. Most frameworks retrofit agent support; HyperFrames treats it as the foundational constraint.

Whether or not this particular framework becomes dominant, the design choices — HTML-first, deterministic rendering, slash command integration, thin core with a Frame Adapter extension pattern — are worth studying. They represent a coherent vision of what agent-native tooling looks like.

## Links

- Homepage: https://hyperframes.heygen.com/
- Prompting guide: https://hyperframes.heygen.com/guides/prompting
- Compositions concept: https://hyperframes.heygen.com/concepts/compositions
- Common mistakes: https://hyperframes.heygen.com/guides/common-mistakes
- GitHub: https://github.com/heygen-com/hyperframes

