Front-end15 min read
How I make AI videos with Remotion: a real case study
How I make reels with AI and Remotion: topic, script, React scenes, word-by-word captions and rendering. A real case, with the numbers of Explica, Palito.

I run a trivia account on Instagram where no video was ever filmed or opened in a video editor. Every reel is a React component, rendered frame by frame by Remotion. And most of that code is written by an AI.
The account is Explica, Palito (Portuguese for "Stick explains"): 3D stick figures explain what science has already tested about couples and the human body. One of those videos passed half a million views.
This post walks through the whole flow, from topic to post, with the real code. It also shows where the AI comes in, where it doesn't, and what went wrong along the way.
TL;DR#
- Remotion turns React into video. Each frame is a function of the frame number, and the render comes out as an
.mp4from the command line. - Video as code is great for AI: it doesn't need a graphic editor, just text files and a command to render.
- The flow: topic and study, a script in JSON, a synthetic voice with the timing of each word, a React scene with three.js, preview, render and publishing.
- The AI doesn't draw anything. There's no image or video model: every figure is geometry written in code. The numbers come from the study, and the decisions about topics and format are mine.
- The topic mattered more than anything. On November 6, 2026, couple videos made up about 86% of the account's views.
- Things went wrong too: a reach drop after a political post, a limit of about 25 posts a day and captions that vanished during automated publishing.
What Explica, Palito is#
Explica, Palito is a science reels account, also on TikTok. The videos are in Brazilian Portuguese, and the format is always similar: a question people can check on themselves or on someone they love, a study that tested it and the answer, in 30 to 50 seconds.
The figures live in a low-poly 3D diorama: a round island floating in the air, with the camera circling around it. On top of that come the text plates and a big caption, word by word, synced with the narration.
The channel's method also became an ebook, Palito AI: crie vídeos com IA sem aparecer (in Portuguese). It goes from picking the topic and the hook to reading the numbers after posting. This post sticks to the technical side: how the video is made in code.
Why Remotion#
Remotion is a library that renders React components as video. The core idea is simple: the component reads the current frame number with useCurrentFrame() and returns what should show up in that frame. Remotion opens a headless browser, takes a screenshot of each frame and puts everything together in an .mp4 with the audio.
For this project, that solved three things:
- Everything is reusable. Island, figure, plate, caption and transition become components. A new video is mostly a combination of pieces that already exist.
- 3D without leaving React. With the
@remotion/threepackage, the scenes use three.js through React Three Fiber, and the camera is also just a function of the frame. - An AI can operate it. There's no timeline to drag and no button to click. The video is a
.tsxfile, and the render is a terminal command. That's exactly the kind of work a coding agent is good at.
The flow, from topic to post#
I work with Claude Code open in the project folder. It has a skill with the channel's rules (how to pick a topic, how to write the hook, which pieces to use) and a production guide. Every video follows the same steps:
1. Topic and study#
The topic goes through a single filter: can people check this on themselves or on someone they love, right now? "Do couples end up looking alike?" passes. This week's news doesn't.
Once the topic is chosen, the AI looks for the primary study (the paper in the journal or on PubMed, not the news story about it) and writes down author, year, journal, sample and what was measured. If it can't find the primary source, it changes the topic. For an observational study, the video talks about association, not cause.
2. A script in JSON#
The script is a narration.json: a list of lines, each with an id, the text the voice will read and, in the first one, the title that shows up on screen. This is the real script of the hug video (in Portuguese, like the account):
[
{ "id": "hook", "say": "Um abraço depois da briga mexe com o seu humor até no dia seguinte.", "caption": "O abraço que [SALVA O DIA SEGUINTE]" },
{ "id": "mito", "say": "Parece só um carinho. Coisa pequena.", "caption": "" },
{ "id": "mas", "say": "Mas pesquisadores ligaram pra quatrocentas e quatro pessoas, toda noite, por duas semanas.", "caption": "" }
]The hook says a hug after a fight affects your mood until the next day. Numbers are spelled out ("quatrocentas e quatro", four hundred and four) because a synthetic voice reads them, and it makes fewer mistakes that way.
3. The voice and the timing of each word#
The narration comes from edge-tts, a Python library that uses Microsoft Edge's neural voices. The detail that makes the captions work: besides the audio, it returns when each word starts.
for line in lines:
path = out_dir / f"{line['id']}.mp3"
words = []
with open(path, "wb") as f:
async for chunk in edge_tts.Communicate(line["say"], VOICE, rate=RATE, boundary="WordBoundary").stream():
if chunk["type"] == "audio":
f.write(chunk["data"])
elif chunk["type"] == "WordBoundary":
# offset comes in 100 ns units: turn it into a frame at 30 fps
words.append([round(chunk["offset"] / 1e7 * FPS), chunk["text"]])
sec = duration(path) # ffprobe
timings[line["id"]] = {"voice": math.ceil(sec * FPS), "frames": math.ceil((sec + GAP) * FPS), "words": words}The result is a timings.json with how many frames each line lasts and a list of [frame, word]. Everything in the video syncs to it.
4. The scene in React#
Each line becomes a shot, with the camera moving from one point to another while the voice speaks. The hug video has eight:
This is the shot where the hug video delivers the result, with the names translated to English (in the project they're in Portuguese) and the colors removed to keep it short:
import { Cam, Plates, Shot, Source } from '../v2/kit'
import { makeWordFrame } from '../shared'
import timings from './timings.json'
// at('payoff', 'baixo') = frame where the narrator says that word
const at = makeWordFrame(timings)
// Hug (the couple hugging) is declared above, in the same file
const Payoff: React.FC<SceneProps> = ({ parts }) => (
<>
{/* camera travels from a to b during the line and shakes on the strongest word */}
<Shot parts={parts} palette="sun" a={Cam(0.4, 14, 2.4)} b={Cam(0.2, 10, 2.6)} shake={at('payoff', 'baixo')}>
<Hug />
</Shot>
<Plates
items={[
[at('payoff', 'seguinte'), 'NO DIA SEGUINTE', 'big'],
[at('payoff', 'baixo'), 'humor ruim mais baixo'],
]}
/>
<Source text="Murphy et al., 2018 · PLoS ONE" delay={20} />
</>
)Notice that nothing there mentions seconds. The camera, the plates and the screen shake are tied to words in the narration ("baixo" means lower, "seguinte" means next). If the voice changes speed, timings.json changes and the video stays in sync.
makeWordFrame is small. It looks up the word in the list and returns the frame:
export const makeWordFrame = (timings: Timings) => (lineId: string, word: string) => {
const words = timings[lineId].words
const clean = (text: string) => text.toLowerCase().replace(/[^\p{L}\d]/gu, '')
const hit =
words.find(([, text]) => clean(text) === clean(word)) ??
words.find(([, text]) => clean(text).startsWith(clean(word)))
return hit ? hit[0] : 0
}5. Word-by-word captions#
The big caption uses the same list. On every frame, the component finds the last word that has already been spoken and draws only that one, with a pop-in. Words with one or two letters stick to the next one, so the caption doesn't flicker.
6. Preview and render#
A script ties the steps together. Without --render, it generates a strip of frames in a PNG, which the AI opens and looks at before spending minutes rendering. That's how it catches cut-off text, a plate covering a face or an empty scene.
./make.sh abracobriga AbracoBriga 120 400 700 # preview strip with 3 frames
./make.sh abracobriga AbracoBriga --render # final mp4 in out/upload/The render creates a composition just for that video and calls npx remotion render. Then ffmpeg normalizes the volume and makes a lighter version, under 10 MB, to upload from the browser. The hug video, at 41 seconds, has 1,225 frames at 30 fps.
7. Publishing#
Publishing goes through the AI too, with Claude in Chrome driving Instagram on the web: upload, crop, caption with the source and a scheduled time. It's the most fragile step of the flow, and I'll explain why further down.
Where the AI comes in and where it doesn't#
| Step | Who does it |
|---|---|
| Account focus, format and how many posts a day | me |
| Finding the study and checking the numbers | AI, in the primary source |
| Script, post caption and scene code | AI, following the skill |
| Voice | synthetic voice (edge-tts) |
| Drawing, animation and camera | code: React, three.js and Remotion |
| Looking at the preview and fixing it | AI |
| Login, password and decisions about the account | me |
What I like most about this setup: there's no AI-generated image. Every figure is a cylinder, a sphere and four sticks in three.js. The same code rendered twice gives the same video, and no number shows up on screen without coming from a study someone actually opened.
The AI's job is that of a very fast teammate: it researches, writes, builds the scene with the pieces that exist, looks at the result and fixes it. In one late night on November 3, 23 new videos came out that way. The rules it follows, though, came from me and from the numbers.
The numbers#
The account's Insights don't open in the browser, so the numbers below are the public ones, read on the profile, each with the date it was read.
- On November 3, 2026: 70 reels, about 1.1 million combined views and 710 followers. The looking-alike video had 466 thousand views, 44 thousand likes and 332 comments. Couple videos were about 87% of all views.
- On November 6, 2026: 114 posts, about 1.67 million views and 1,069 followers. The looking-alike video reached 598 thousand, and couple videos still made up about 86% of the views.
- On November 9, 2026: 1,272 followers. For comparison, on October 28 the account had 43.
The rest of the account tells another story. Gym videos had a median of 1.9 thousand views, with one exception ("Why won't my calves grow?", 40.8 thousand on November 3). General trivia, history and news stayed around 1.5 thousand.
And views didn't turn into followers at the same rate: between November 3 and November 6, 570 thousand more views brought only 359 followers, about 0.06%. The top of the funnel works; the middle doesn't yet.
What went wrong#
The drop on October 28#
Until October 28, almost every reel stopped between 1.2 and 1.9 thousand views. That night, from 7 p.m., distribution collapsed: a video about the ocean got 179 views, a fact-check of a fake political story got 28, and the next morning's video had 4 views after 1 hour and 40 minutes.
The account status was all green, with no formal restriction. My hypothesis is the combination of a political post with 5 posts a day. I can't prove it, but I changed both: I dropped news and politics from the topics and went down to 3 posts a day for a while. A few days later came the couple videos.
The limit of 25 a day#
When the couple videos took off, I raised the volume. One night, after about 24 scheduled posts and 2 posts on the same day, Instagram started answering "We couldn't schedule your post". That matches a limit of about 25 posts created in 24 hours that I ran into in practice. Moving a post that's already scheduled doesn't count; only creating one does.
The finished videos went into a list with captions and suggested times, and were scheduled the next day.
The caption that disappeared#
Pasting the caption and clicking Share through JavaScript left reels with no caption, even with the field and the character counter looking right on screen. What worked: typing line by line, pressing a real Enter between them and really clicking the button. After publishing, the AI reloads the reel and checks that the caption is there.
The wrong account#
With more than one Chrome connected, a video went out on the wrong account twice. Today, before any post, the AI checks the name of the logged-in account on the screen itself and stops if it isn't Palito's.
What I learned about hooks#
On November 5, I asked to standardize the hook of the videos that went most viral. It became a formula (translated here):
"You know when [everyday situation]? It's not [popular excuse]. Science tested it."
With the on-screen title in the format "Why [who] [KEYWORD?]" and the ending "Tag your love and answer: [a dispute between the two of you]?".
It worked as a base, but not as magic. The next videos stopped at a median of 1.9 thousand views, and by November 10 the channel had fallen into sameness: same hook, same five static scenes, distant camera. That's where format 2.0 came from, inspired by the short 3D explainers that go viral abroad: eight short shots, a camera that's always moving, a twist with "But" and at least one cut to inside the body.
The first videos in 2.0 are from November 16, 2026. It's too early to say whether they worked: I compare the median views after 72 hours, and that reading is due on November 19.
What I learned#
The topic matters more than the production. The looking-alike video is in the old, simpler format. What it has is a question a couple can check on themselves, right away. "Abstract" couple topics, with a good but distant study, didn't catch on (1.4 and 1.7 thousand views).
Don't judge a reel in 24 hours. "Sleeping close" had 2.8 thousand views on November 3 and passed 120 thousand three days later.
Video as code pays off through reuse. The first 3D video took time. Once the kit was ready (island, figure, plate, caption, camera), a new video is a script plus a scene, and the scene is almost all composition.
Sync by word, not by second. Tying each effect to a word in the narration is what let me change the voice speed without redoing a single scene.
Write the rules for the AI, not just the request. The skill with what worked, what didn't and the measurements that avoid rework (where the title covers the scene, how far the camera goes) is worth more than any well-written prompt.
Things to watch out for#
- Remotion's license. It's free for individuals and small companies; larger companies need a paid license. Check the license terms before using it at work.
- AI label. Instagram asks that content with a realistic AI-generated voice be labeled as such. Read the platform's current rule before publishing.
- Health and science. No doses, no prescriptions, no number that isn't in the study. When the study is observational, the video says it doesn't prove cause.
- Real images only with credit. When a video mixes in a real photo, it comes from a collection with an open license (Wikimedia Commons, NASA), with the credit on screen.
- Social media automation breaks. The interface changes, a button gets renamed, a limit shows up. Check every post after it goes out.
- Memory and machine. 3D rendering is heavy. I run it with
--concurrency=2so the machine can handle it.
References#
Read next

Front-end16 min read
Multilingual SEO in React: hreflang, prerendering, sitemap

Front-end15 min read
How to share passwords securely: inside Yush

Front-end13 min read
Vite 8 vs Vite 7 with Rolldown: I benchmarked my build