diff --git a/skills/remotion-to-hyperframes/SKILL.md b/skills/remotion-to-hyperframes/SKILL.md index 7d054d6bb..9f43af4fc 100644 --- a/skills/remotion-to-hyperframes/SKILL.md +++ b/skills/remotion-to-hyperframes/SKILL.md @@ -7,26 +7,95 @@ description: Translate a Remotion (React-based) video composition into a HyperFr ## Overview -Translate Remotion (React-based) video compositions into HyperFrames (HTML + GSAP) compositions. Most Remotion idioms have direct HyperFrames equivalents — the translation is mechanical for ~80% of typical compositions. This skill encodes the mapping and guards against the lossy 20%. +Translate Remotion (React-based) video compositions into HyperFrames (HTML + GSAP) compositions. Most Remotion idioms have direct HyperFrames equivalents — the translation is mechanical for ~80% of typical compositions. This skill encodes the mapping and guards against the lossy 20% by refusing to translate patterns that don't fit HF's seek-driven model and recommending the runtime interop pattern from [PR #214](https://github.com/heygen-com/hyperframes/pull/214) instead. + +The skill ships with a **tiered test corpus** (T1–T4, 4 fixtures total) that grades translations against measured SSIM thresholds. Don't translate without running the eval — a translation that "looks right" but renders 0.05 SSIM lower than the validated baseline is silently wrong. ## Workflow -1. **Lint the source.** Run the source-lint script against the Remotion project to surface any patterns that can't translate cleanly (React state hooks, async metadata, third-party React components). If the source uses any blocker pattern, recommend the runtime interop escape hatch (PR #214 pattern) instead of attempting a translation. +### Step 1: Lint the source + +Run [`scripts/lint_source.py`](scripts/lint_source.py) over the Remotion source directory. The lint detects patterns that can't translate cleanly: + +- **Blockers** (refuse + recommend interop): `useState`, `useReducer`, `useEffect`/`useLayoutEffect` with non-empty deps, async `calculateMetadata`, third-party React UI libraries (MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI). +- **Warnings** (translate after dropping the construct): `@remotion/lambda` config, `delayRender`, `useCallback`, `useMemo`, custom hooks. +- **Info** (translate with note): `staticFile`, `interpolateColors`. + +If any blocker fires, **stop**. Read [`references/escape-hatch.md`](references/escape-hatch.md) and surface the recommendation message. Warnings don't stop translation — drop the offending construct in step 3 and note the gap in `TRANSLATION_NOTES.md`. `@remotion/lambda` config is the canonical warning case: the skill drops the import + `renderMediaOnLambda(...)` calls but translates the rest of the composition. + +### Step 2: Plan the translation + +Read [`references/api-map.md`](references/api-map.md) — the index of every Remotion API and its HF equivalent or per-topic reference. Identify which topic references you'll need based on what the source uses: + +| Source contains | Load reference | +| ------------------------------------------------------------------------- | --------------------------------------------- | +| `Composition`, `defaultProps`, `schema`, `calculateMetadata` | [`parameters.md`](references/parameters.md) | +| `Sequence`, `Series`, `Loop`, `AbsoluteFill`, `Freeze` | [`sequencing.md`](references/sequencing.md) | +| `useCurrentFrame`, `interpolate`, `spring`, `Easing`, `interpolateColors` | [`timing.md`](references/timing.md) | +| `Audio`, `Video`, `Img`, `IFrame`, `staticFile`, `delayRender` | [`media.md`](references/media.md) | +| `TransitionSeries`, `@remotion/transitions` | [`transitions.md`](references/transitions.md) | +| `@remotion/lottie` | [`lottie.md`](references/lottie.md) | +| `@remotion/google-fonts/`, `Font.loadFont`, `@font-face` | [`fonts.md`](references/fonts.md) | + +Don't load all of them — load only what the specific source needs. + +### Step 3: Generate the HF composition + +Emit `index.html` with: + +- Root `
` carrying the composition's `data-composition-id`, `data-start="0"`, `data-duration` (in seconds), `data-fps`, `data-width`, `data-height`, plus one `data-*` per scalar prop. +- A flat list of scene divs with `data-start` / `data-duration` / `data-track-index`. +- Inline ` + + +
+
+
HELLO
+
+ + +
+ + diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/package.json b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/package.json new file mode 100644 index 000000000..1c33b63b4 --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/package.json @@ -0,0 +1,14 @@ +{ + "name": "tier-1-title-card-remotion", + "version": "0.0.0", + "private": true, + "scripts": { + "render": "remotion render TitleCard out/baseline.mp4" + }, + "dependencies": { + "@remotion/cli": "^4.0.0", + "react": "^18.3.1", + "react-dom": "^18.3.1", + "remotion": "^4.0.0" + } +} diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/remotion.config.ts b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/remotion.config.ts new file mode 100644 index 000000000..a22cafc5e --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/remotion.config.ts @@ -0,0 +1,13 @@ +import { Config } from "@remotion/cli/config"; + +// Match HyperFrames' default render so SSIM diffs measure translation +// fidelity, not encoder differences. +// +// setVideoImageFormat("png") avoids the JPEG limited-range/full-range +// colorspace flag (yuvj420p vs yuv420p) that otherwise costs ~0.05 SSIM. +// +// setColorSpace("bt709") matches HF's BT.709 SDR output. +Config.setVideoImageFormat("png"); +Config.setColorSpace("bt709"); +Config.setOverwriteOutput(true); +Config.setConcurrency(1); diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/Root.tsx b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/Root.tsx new file mode 100644 index 000000000..f8dd72770 --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/Root.tsx @@ -0,0 +1,13 @@ +import { Composition } from "remotion"; +import { TitleCard } from "./TitleCard"; + +export const RemotionRoot = () => ( + +); diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/TitleCard.tsx b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/TitleCard.tsx new file mode 100644 index 000000000..924557fff --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/TitleCard.tsx @@ -0,0 +1,34 @@ +import { AbsoluteFill, interpolate, useCurrentFrame } from "remotion"; + +export const TitleCard = () => { + const frame = useCurrentFrame(); + + // Fade in 0-15, hold 15-75, fade out 75-90. + const opacity = interpolate(frame, [0, 15, 75, 90], [0, 1, 1, 0], { + extrapolateLeft: "clamp", + extrapolateRight: "clamp", + }); + + return ( + +
+ HELLO +
+
+ ); +}; diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/index.ts b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/index.ts new file mode 100644 index 000000000..f31c790ed --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/src/index.ts @@ -0,0 +1,4 @@ +import { registerRoot } from "remotion"; +import { RemotionRoot } from "./Root"; + +registerRoot(RemotionRoot); diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/tsconfig.json b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/tsconfig.json new file mode 100644 index 000000000..eb6f0f806 --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-1-title-card/remotion-src/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2018", + "module": "ESNext", + "jsx": "react-jsx", + "strict": true, + "esModuleInterop": true, + "moduleResolution": "node", + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "isolatedModules": true + }, + "include": ["src"] +} diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-2-multi-scene/.gitignore b/skills/remotion-to-hyperframes/assets/test-corpus/tier-2-multi-scene/.gitignore new file mode 100644 index 000000000..def72ea01 --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-2-multi-scene/.gitignore @@ -0,0 +1,14 @@ +# Generated by setup.sh +remotion-src/public/ +hf-src/assets/ + +# Remotion / HF dependencies +remotion-src/node_modules/ +remotion-src/package-lock.json + +# Render output +remotion-src/out/ +hf-src/out/ +hf.mp4 +diff/ +strip/ diff --git a/skills/remotion-to-hyperframes/assets/test-corpus/tier-2-multi-scene/README.md b/skills/remotion-to-hyperframes/assets/test-corpus/tier-2-multi-scene/README.md new file mode 100644 index 000000000..17659a683 --- /dev/null +++ b/skills/remotion-to-hyperframes/assets/test-corpus/tier-2-multi-scene/README.md @@ -0,0 +1,54 @@ +# Tier 2 — title-image-outro + +## What it tests + +Three-scene composition. Each scene exercises a different Remotion idiom: + +1. **Scene 1 (0–2 s)** — TitleScene with `spring({damping:12, stiffness:100, mass:1})` + driving a `transform: scale()` on text. Tests the lossy `spring → GSAP ease` translation. +2. **Scene 2 (2–4 s)** — ImageScene that fades in a `staticFile`-loaded image and + linearly scales it from 0.8 → 1.0. Tests asset paths + linear `interpolate`. +3. **Scene 3 (4–6 s)** — OutroScene with a 1-s linear fade-in. Sanity check after + the harder scenes. + +A silent 6-second WAV plays throughout at `volume={0.5}`. Tests `