Skip to content
How I Built a Murder Mystery Game With a Team of Claude Agents
Build your own software

How I Built a Murder Mystery Game With a Team of Claude Agents

A team of Claude Opus 5.5 agents wrote, cast, voiced, built and tested a browser murder mystery over eight days. Here is every step, every version, what went wrong and what it cost.

36 min read

0

I gave a team of Claude Opus 5.5 agents one goal, build a murder mystery game that runs in the browser, and over eight days they wrote the story, cast and voiced the characters, built every room in Blender and coded the whole thing.

The game is called Dead Air on the Meridian, it took twelve versions to get right, and at API prices the Claude work would have cost about $4,200.

This post walks through how it was built, step by step, including the parts that broke, and it goes a lot deeper than the video does.

Ad disclosure: Fish Audio sponsored the video this post is based on and added credit to my API balance. Every voice in the game was made with Fish Audio Drama 3.

Dead Air on the Meridian, the finished gamePlay the game
Nora Pell sitting down for her interview in the dining car

You can play it in your browser from the card above, and the making-of site at meridian-making-of.vercel.app has every file and before-and-after from the build, which I link to throughout.

What the game is

You play a detective on a 1936 sleeper train that is stuck overnight in a snowbound pass, and Desmond Crane, a radio host whose whole show was exposing other people's secrets, has been killed in his compartment.

You search his room for clues, and anything useful goes into a notebook that keeps track of the people you have met, the evidence, what everyone told you and a timeline of the night.

Then you interview the suspects, every one of whom is hiding something, and when a clue proves someone is lying you show it to them and their story falls apart.

When you think you know who did it, you make an accusation and back it up with proof, and the ending you get depends on who you accuse and whether you can prove it.

What it cost

I am on the Claude Max plan, so I paid my normal subscription across a few $200 Max accounts, but I had the agents add up exactly what the same work would have cost at Anthropic's API prices.

Cost
Claude, at API prices (paid by subscription)$4,196
Fish Audio Drama 3, every voice in the gameAbout $1.40
Rented GPUs on RunPodAbout $25
Time, first activity to last8 days 19 hours

Almost all of the Claude figure is Opus 5.5, at about $4,140, with about $55 of Sonnet 5.5 and under a dollar of Haiku 4.5.

The agents used 9.7 billion tokens in total, and 96% of those were cache reads, which is the model re-reading the project's files and its own earlier work at a much lower rate.

Part of the buildCostShare
Building the game, versions 1 to 12$2,51760%
Making the characters realistic$76418%
The making-of site$48712%
Coordinating everything else$42810%

Agents produced replies during 71 hours of those eight days, and because several ran at once, the total adds up to 202 agent-hours.

This is a fairly static game, since you can't walk freely around the train and nothing is generated on the fly, so most of the spend went on trial and error that the skill below should now save you.

The Game Studio skill

I started by asking Claude how we could build an entire game studio run by Claude agents, so that this game and any future game would have a base to work from.

Claude ran a dynamic workflow that spawned 20 research agents, and they studied how real game studios work: who does what, the best practices, the terminology and the rules each department follows.

That research became Game Studio, a free Claude Code plugin with 15 agents, where each agent is one job at a real studio with its own job description and a written quality bar.

Game Studio on GitHub

The free Claude Code plugin with all 15 agents, the Blender scripts and the example murder mystery.

Open
Install the plugin
claude plugin marketplace add per-simmons/claude-game-studio && claude plugin install game-studio@game-studio
Install the plugin
claude plugin marketplace add per-simmons/claude-game-studio && claude plugin install game-studio@game-studio

The 15 jobs

AgentWhat it does
ProducerPlans each round, breaks it into tickets, runs departments in parallel and routes every review
Creative directorOwns the vision and the pillars, and runs the fun audit
ResearcherFinds how other games solved the same problem, with a source for every claim
Writers' roomDrafts the cast, secrets, the true timeline, clues, lies and endings
Narrative leadReviews every story draft for logic, fairness and voice
Story auditorAttacks the finished story to find holes, in three different modes
Systems designerDesigns the rules, the clock and the guardrails that stop suspects being tricked
Art directorOwns the look and reviews every image and 3D scene
Character artistPaints every character, their turnarounds and their mood portraits
UI designerDesigns the screens, the notebook and the captions
Voice directorCasts the voices and writes an acting direction for every line
EngineerBuilds the game itself, with tests and automated playtest bots
QA leadPlays the game with bots and AI testers and files bugs back to the engineer
Blender artistBuilds the rooms and props in Blender by script
Render wranglerRuns the heavy renders on rented GPUs with hard cost limits

The last two are optional "pro" departments, and the free path makes 2D art with a placeholder voice and needs no API keys at all.

Nobody approves their own work

All 15 agents work off one shared ticket board, where every ticket has an owner, a status and a reviewer who is always a different agent.

The writers' room is reviewed by the narrative lead, the character artist, Blender artist and UI designer are reviewed by the art director, the researcher and systems designer are reviewed by the creative director, and QA files bugs back to the engineer.

The studio also stops at four checkpoints, called gates, where only you can approve, change or kill the project: concept, vertical slice, content complete and polish.

The prompt and the plan

You start a game with one command and a rant about the game you want, and it doesn't need to be polished, because the producer asks a few follow-up questions about the premise, the length and which path you want.

The prompt that started the game
/game-studio:new I want to build a murder mystery game that runs in the browser. You're on a 1936 sleeper train stopped overnight by a snow slide, someone's been murdered, and you have to interview six suspects before the sheriff arrives at dawn. Every suspect is voiced, so you catch liars by how they sound. Make it look cinematic, with real 3D rooms.
The prompt that started the game
/game-studio:new I want to build a murder mystery game that runs in the browser. You're on a 1936 sleeper train stopped overnight by a snow slide, someone's been murdered, and you have to interview six suspects before the sheriff arrives at dawn. Every suspect is voiced, so you catch liars by how they sound. Make it look cinematic, with real 3D rooms.

Before any agent builds anything, the studio writes the plan, and it wrote mine in one evening as five documents: the game design, the story, the voices, the look and the tech.

Each document goes to the department that builds that part, and those five files are what every agent worked from for the rest of the build. The making-of site shows the plan at step 2.

The producer then turned the plan into a long build brief with hard rules, such as never stealing focus from my screen, never running heavy work on my Mac while I was recording, and never naming the AI tools inside the game.

From there my only job was to play each version and write down what was wrong, which is how the twelve versions further down came about.

The story team

A murder mystery falls apart if the story has holes in it, so the story gets more checking than anything else in the studio, and the making-of site covers it at step 3.

The writers' room drafts the whole case: the cast, each person's secret, a minute-by-minute timeline of what really happened that night, every clue and lie, the red herrings, the choices you can make and the endings.

The narrative lead then checks the draft for logic, cleverness and fairness, using the classic fair-play rules of detective fiction, where the reader must be able to see every clue the detective uses, and the two go back and forth until it approves.

Three auditors try to break it

Once the narrative lead is happy, three story auditors attack the story, and each one has a different job and returns a ranked list of fixes with the exact replacement text.

The first is a speed-solver, which tries to crack the case as fast as possible and looks for shortcuts, to find out whether the game is too easy or can be cheated.

The second reads like a story editor, looking for plot holes, motives that don't add up and moments where a character knows something they shouldn't.

The third is a consistency auditor, which checks that every time, place and fact lines up everywhere it appears.

The auditors never edit the story themselves, because everything they find goes back to the narrative lead to fix, and that keeps one agent responsible for the finished version.

Bots check the game is fair

The engineer later turns the story into a game and runs two bots against it, and both have pass marks written into the build brief.

A speedrun bot has to find that the shortest path to the true ending still takes at least 16 actions and 90 minutes of game time, and a brute-force bot that plays 10,000 random games must reach the true ending less than 1% of the time.

Once the story holds up, it becomes the narrative bible, the finished story that every other agent works from, and nobody downstream is allowed to change the plot.

The voices, with Fish Audio Drama 3

The voice director's job is casting, and because none of these characters exist, every voice had to be designed from scratch and could never be a clone of a real person.

Its prompt also says every character needs a tell in their voice, even the honest ones, because in a murder mystery you should be able to hear it when someone might be lying. You can read that prompt at step 4.

I used Fish Audio, who sponsored the video, and their newest model, Drama 3, which they describe as their most emotive voice model yet.

Voice Design and text to speech

Fish Audio gives you two ways to make a voice, and it helps to know the difference before you start.

Text to speech is where you pick a voice that already exists from their library, type a line, and it reads the line out in that voice.

Voice Design is where you create a brand-new voice by describing it in plain words, and it gives you three takes to choose from, so every voice in this game was made with Voice Design.

Voice Design description for Briggs, the conductor
A steady Midwestern American railroad conductor around sixty, gravelly baritone, plain-spoken, dry humour.
Voice Design description for Briggs, the conductor
A steady Midwestern American railroad conductor around sixty, gravelly baritone, plain-spoken, dry humour.

The studio did this through the Fish Audio API rather than the dashboard, so it sent 15 Voice Design requests with three candidates each and kept 13 voices.

My whole setup was creating an API key in the developer tab of the Fish Audio dashboard, putting it in the project's .env file and telling Claude to read the documentation, and after that I never touched the dashboard.

Acting directions on 628 lines

The first versions used the browser's built-in voice, and it read every line completely flat, which is no good when the game's main skill is reading a suspect's voice.

So the voice director went through all 628 lines and wrote an acting direction in square brackets for each one, the way you would direct a real voice actor, with notes like nervous, voice catching or speaking a little too fast.

A directed line for Briggs
[low, steady, tired] He asked me to wake him the minute there was word on the line. [pause] I knocked. The door wasn't locked.
A directed line for Briggs
[low, steady, tired] He asked me to wake him the minute there was word on the line. [pause] I knocked. The door wasn't locked.

Drama 3 acts on those directions, and a direction carries on until the next one, so where you put it in the line matters.

The making-of site plays the same line in the browser voice, in another AI voice and in Drama 3 at step 4b, so you can hear what the directions change.

Word timestamps drive the captions and the lips

Drama 3 can send back a timestamp for every single word it speaks, and the game uses those timings for two things.

The captions highlight each word as it is said, and the characters' mouths move in time with the words, first on the painted portraits and later on the 3D faces.

Multi-speaker scenes

Normally you generate every line on its own and stitch the lines together, which makes a conversation sound like separate recordings sitting next to each other.

With Drama 3, I sent one request with the whole script and four speakers, the Colonel, Briggs, Nora and me, and it gave me the entire conversation in a single take.

Every voice stays the same all the way through, and the acting directions still work inside it, like Nora cutting in just a fraction too quickly. You can play that scene at step 4c.

Fixes the voices needed

Long pauses. In version 10 the agents measured the silence in all 775 voice takes, found almost 1,000 seconds of it, and made the game skip any pause longer than a quarter of a second.

That cut the gap after "Walter Briggs, conductor" from about a second and a half to a fifth of a second, and everyone now talks a little faster.

One character, two voices. Celeste's voice changed partway through some lines, because the game split those lines in two where her accent slips.

The fix kept her in one voice, moved the accent slip into an acting direction, and re-rendered 29 of her lines in Drama 3.

New lines late in the build. Version 9 gave the detective a spoken thought on every clue, which took 127 new or re-recorded lines, and version 12 added 16 greeting lines so the detective says hello as each suspect sits down.

Even with all those extra takes and re-renders, voicing the whole game cost about $1.40.

The character artist

An image generator draws a slightly different person every time you ask, so the character artist's job is to lock down exactly what every character looks like, and the art director checks every image it makes.

It paints each character from a written description with the ChatGPT image generator through my subscription, and the art director sends back the ones that miss.

Celeste Varga's turnaround sheet from the front, side and back
Celeste's turnaround sheet, which every later image of her has to match.

Two drafts of Celeste, the film star, were sent back before the third one was approved, and once a character is approved it gets a turnaround sheet of the front, the side and the back so they look the same in every image after that.

Because this is a mystery, it also paints a portrait for each mood, so a suspect looks different when they are calm and when they are cornered, and Celeste got four: composed, guarded, cracking and broken.

Celeste's four mood portraits
Celeste's mood portraits. They came out too subtle, and a more animated set would have given the player more to read.

I think those moods ended up too similar, because there is not much difference between cracking and broken, and I would push the agents for bigger changes next time.

Mouth shapes, then talking-head models

To make the painted portraits talk, the character artist first redrew each one with a different mouth shape, such as an O, an E and closed lips, and the game swapped in the matching mouth for the letters in each word as it was spoken.

It worked, but it looked like a puppet, so the studio tested ten open-source talking-head models, which take one portrait and one voice line and make a video of that face actually saying it.

The candidates included FlashHead, MuseTalk, EchoMimic, LeapTalk, Sonic, InfiniteTalk and Hallo3, and the agents ran most of the tests while I reacted to the results.

FlashHead won, and it made 722 talking clips for about six dollars of rented computer time. You can compare the models at step 5e.

Eventually the game moved past portraits completely into real 3D characters, and FlashHead stayed in the skill as the right tool for any game built on 2D portraits.

Building the rooms in Blender

I wanted the rooms to look like a real 1930s train and to be fully 3D, so the Blender artist builds every room in three stages: the mesh, the materials and the light.

The mesh

A mesh is the 3D shape of everything in the room, made out of thousands of tiny flat triangles joined together, a bit like an unpainted model kit where every wall, seat, table and window is there but nothing has colour yet.

Opus 5.5 already knows how big a real 1930s railway car is, about three metres wide with tables about 74 centimetres high, so it placed every object at real sizes from what it already knew.

It never opens Blender and clicks around, because it writes Python scripts that run Blender in the background with no window, which works like controlling an app through an API.

That uses far fewer tokens than an agent looking at screenshots of an app, and the scripts are reusable, so a lot of them are in the Game Studio skill along with instructions for getting your own agents running Blender.

Materials and light

The materials are what make wood look like wood and velvet look like velvet, and then the agent adds the lights: the lamps, and the moonlight coming through the windows.

Working out what all those lights do takes Blender a few minutes per picture, while a game has to draw sixty pictures every second, so Blender works the light out once and saves it into the colours of the room itself.

That is called baking, and it means the game does no lighting of its own for the room, because it loads the room with the light already on it.

The dining car rendered in Blender
The dining car as Blender renders it. The live game version is a little softer, because the saved light has to stay small enough to download quickly.

You can drag between the bare clay room and the lit room at step 6b, and the stages go from clay, which is only the mesh, to baked, which has the light saved on, to the final version you see in the game.

The live room started at about 60% as good as Blender's picture, and a polish pass in version 8 added shine on the brass, silver and glass, a soft haze in the lamp light and snowy mountains outside, which got it to about 80% by the art director's estimate.

Renting GPUs on RunPod

These renders were maxing out my Mac Studio, so the agents rented graphics cards by the hour on RunPod and ran the heavy work there.

The setup is adding money to a RunPod account, creating an API key and giving it to the agent, which then picks a suitable machine and shuts it down afterwards.

The build brief capped render jobs at 40 cents an hour and 45 minutes each, and every agent logged its GPU spend, which is how I know the whole build came to about $25. I'd guess it would have taken another week on my own machine.

The rented machines were also how the agents kept working while I was recording, because the brief banned heavy local work whenever OBS was running.

Why the game runs on Three.js and WebGL

The engineer built the game with Three.js, a free toolkit for drawing 3D in a web page with your computer's graphics chip, and the making-of site explains the choice at step 7.

There were really three ways to get a game like this running, and each one has a real strength.

StrengthProblem for this game
Unity or UnrealThe most realistic, with tools like Unreal's MetaHuman for charactersA heavy desktop app the agents have to click around, and players have to download the game
Rust, WebAssembly and WebGPUThe fastest option inside a browserMuch more to build from scratch, and WebGPU isn't on every device yet
Three.js and WebGL (what I used)Runs in any browser, and agents can build and test it themselvesLess detail available than WebGPU on the same chip

Unity and Unreal have MCP servers, so agents could build in them through code, but testing would mean driving a huge desktop app that burns through tokens and struggles even on a Mac Studio, and I wanted the game playable in a browser tab.

Rust is a programming language that is becoming popular for AI-built software, and FilmCraft, the Premiere Pro clone I made a video about, is written entirely in it.

WebAssembly is what lets code like that run inside a web page at close to full speed, and WebGPU is the newer way for a web page to use your graphics chip, getting more out of it than WebGL, the older standard.

I went with Three.js on WebGL because it is lighter and it is what agents know best right now, so they could build the game and play-test it in a real browser themselves.

The game also suits it, because it has fixed rooms whose lighting is worked out ahead of time, and a big open world or a level generated on the fly is where Rust and WebAssembly would make the most sense.

I tested whether WebGPU would look better

I had an agent build a fair side-by-side, with the same camera, the same lights, the same character pose and the same 3D files, rendered four ways in the dining car.

A straight port to Three.js's WebGPU renderer looked almost identical to the live WebGL game, because the look comes from the baked rooms and the custom skin, eye and hair shaders, and none of that depends on the renderer.

Turning on WebGPU extras like softer shadows and smoother edges cleaned up the hair edges a little, but it also made the face slightly blurry, so it wasn't an upgrade.

A Rust version built with the Bevy engine looked clearly worse, with flatter lighting, black windows and a jagged hairline, and it was a 101 MB download that took 15 to 18 seconds to load.

Every version held 60 frames per second, and the agent estimated a full WebGPU port at two to four weeks for parity, so I stayed on WebGL.

The engineer also squeezes every 3D file down so it downloads quickly, and a small unpacker in the browser opens them as the game loads.

The 3D characters

The characters were the hardest part of the whole build by a long way, and they took most of the versions.

I'll use Briggs, the conductor, as the example, because he is the one we got furthest with, and the making-of site walks through every stage of him at How the conductor got real.

From painted portrait to claymation

In the first versions Briggs wasn't 3D at all, because he was a painted portrait that FlashHead made talk.

The first 3D characters, in version 4, were made by turning the painted characters into 3D models on rented GPUs and rigging them in Blender so they could walk, sit and talk, and they looked like claymation.

Briggs as the first claymation 3D version
The first 3D Briggs, which I described as claymation. It had a seam down the middle of his face, with each half a different skin tone.

It looked like something out of Wallace and Gromit, which might have been good enough, but I wanted to see how realistic agents could get.

Version 12 rebuilt everyone on MPFB, a free, open-source character maker for Blender, which fixed the seam and gave them real eyes, teeth and five-finger hands, but the faces were still smooth and plastic and didn't look like anyone in particular.

Photo to 3D head

For the face, the agents generated realistic photos of each character from their concept art, then used Rodin, an AI model that turns photos into a 3D head, to build a head from those photos.

They also tried TRELLIS.2, another photo-to-3D model, but it turned the concept art into a caricature and the photos into a soft blob, so Rodin won.

A generated head can't talk on its own, because it has no rig or face controls, so the agents wrapped the MPFB head onto the Rodin shape, getting 98% of the face within two millimetres while keeping every control it needs to move.

Rodin narrowed every nose by about 6 to 12%, consistently across attempts, so the agents wrote a script that measures the nose against the photo and widens it back.

Skin and eyes

For the skin, the agents projected the real photo onto the face from five angles, so the skin has real colour and detail, and added scanned skin pores on top.

Skin looks like plastic in most browser games because, as the research report pointed out, only about 6% of the light that comes back from skin bounces off the surface and the rest scatters underneath first.

So the agents wrote a skin shader that lets light go slightly into the skin, plus light that bleeds through the ears and nostrils, and they lit each character with light sampled from the room so faces match their surroundings.

The eyes got a four-part shader with a clear, curved cornea that bends light the way a real one does, an iris with depth, a dark ring around it and a wet line at the lid, so they look wet and catch the light.

Talking with Audio2Face

FlashHead only works on flat 2D pictures, so for the 3D characters the agents switched to Audio2Face, a free model from NVIDIA.

It listens to every Fish Audio line and works out how the face should move to say it, using 52 controls for things like an open jaw or pursed lips, so the lips match the words.

Those 52 controls are the standard set Apple uses for face tracking, which is why every character has to keep them through all the rebuilding.

The agents ported Audio2Face to plain Python so it runs without an NVIDIA graphics card, and it saves the face movement for each line as a small file the game plays back alongside the audio.

On top of the mouth, a separate layer adds blinks, small eye movements, breathing and eye contact about 75% of the time while a character is speaking, because a face that only moves its lips looks dead.

Hair, the hardest part

The first hair was big, chunky clumps that looked sculpted out of clay.

Real hair is tens of thousands of thin, see-through strands that overlap, and a browser drawing three characters at 60 frames per second can't afford to draw it the way films do.

The usual game shortcut is hair cards, which are flat strips with a picture of hair painted on them, and from close up they show hard edges and gaps where you can see the scalp.

The AI models that build hair from a photo were mostly ruled out, because most of them carry non-commercial licences, and one the agents did try on a rented GPU produced generic bobs that lost to their own work.

In the end the agents grew 20,000 to 40,000 individual strands per character and drew each one as a line a single pixel wide, and in a blind comparison the strands scored 5.5 out of 10 against 2.5 for cards.

Celeste's hair rendering as solid clumps
One of the failed hair versions. Hair stayed the weakest part of every character.

Even then, hair stayed the weakest part of every character, so I cheated and put hats on most of them, because a cap hides the hairline where most of the problems showed up.

Celeste got a free flower sun hat under a Creative Commons licence, and the agents wrote a script that fits a hat to any character's head in about ten seconds.

Clothes

The early clothes were low-detail shapes stuck to the body, and they broke apart into white shards at collars when characters sat down.

Nora got real simulated cloth, with a rib-knit cardigan, a closed blouse collar and a skirt that drapes differently when she is sitting, which is saved as its own shape the game switches to when she sits.

The men's shirt collars were dropped after four rounds, because the fixes held when the characters stood still and broke again whenever they sat.

Why the realistic base characters didn't work at first

I also tried pre-made base models, realistic base characters built for games, to see whether a professional starting point would beat what the agents had made.

As they come, the realistic base characters looked like convincing young adults, and they beat our cast on skin detail, eyelids and lashes, wet eyes and the inside of the mouth.

But once the agents wrapped them onto our characters' faces, they lost every blind comparison on likeness: Briggs scored 2.0 against 2.8 for the live version, and Celeste scored 3.2 against 4.2.

The reason is that a wrap moves the surface of the face but keeps the identity underneath, such as the shape of the eyelids, the fat in the cheeks and the fullness of the lips, so Celeste kept reading as a young ingénue and Briggs as a much younger man.

Nora, who is young in the story, was the one character the base actually matched, so she was rebuilt on it.

The renderer had to be fixed before they looked real

Even then, Nora looked far worse in the game than she did in Blender, and an agent spent a whole round working out why, which we called the parity fixes.

She skipped the custom renderer. The exported file was missing one flag, so the game drew her with an old, plain path that had none of the skin, eye or hair shaders.

Her skin maps were plugged into the wrong slots. The export had put a greyscale mask where the skin's surface detail should go, and set the roughness to almost fully matte everywhere, which made her skin look like plastic.

Her eyes were covered. The clear cornea in front of each eye was carrying the white of the eye with no iris, which made her eyes pale and made her look sideways, and a bug in how her two eyeballs were merged pulled them out of their sockets whenever she looked around.

Her hair cut off too much. The hair kept only the solid middle of each strand, so the scalp showed through, and the fix draws hair in two passes, the solid core first and the soft edges blended on top.

Her rig didn't fit. Her body had been fitted to our skeleton while it was in a seated pose, so her suit exploded when she walked, and her mesh turned out to be about 12% smaller than the skeleton with her wrists 13 centimetres off.

The agent ended up rebuilding her on her own original skeleton, renamed so our animations could drive it, and it added a warm fill light on faces and a smaller limit on how far eyes turn, which helped every character.

In a blind review, the fixed Nora scored 6.2 against 3.8 for the live one, the best score of the day, while the other nine characters were rated the same with no regressions.

The lesson for anyone using pre-made characters is to check that they look in your game the way they look in Blender before judging the model itself.

Where the characters ended up

The agents scored every character from 1 to 10, where 7 means a stranger would believe they were watching a real person in a cutscene.

Briggs went from 3 to 6 on the builder's own scores, and nobody reached 7, so the cast looks like a solid game from about ten years ago rather than a current one.

Realistic humans are the hardest thing to make in any game, but for characters that agents built from scratch to run in a browser, I think it is pretty wild how far this got.

Briggs in the final version
Briggs today, after the photo head, the skin and eye shaders, strand hair under his cap and Audio2Face.

Motion

The movement went through its own set of versions, and most of the problems only showed up once the faces were good enough that bad motion stood out.

CMU and Mixamo motion capture

The first characters moved with motion capture from Carnegie Mellon University's free library of recorded human movement, so the walk was a real person's walk, trimmed into a loop about one second long.

Later the agents moved to Mixamo, Adobe's free animation library, where I downloaded clips for sitting idle, sitting and talking, standing and talking, and walking.

The 1.6-second loop

I kept seeing seated characters lift their hands up and down about once a second, and I also noticed characters jumping and hopping on the spot.

The cause was that the seated base pose was a 1.63-second loop that raised the hands on every cycle, the seated talking clip was the standing one, and the standing talk moved the legs and hips enough to look like a hop.

The fix uses the Mixamo clips for the upper body only, on a fixed seated pose, and plays the talking gestures only while a character is actually speaking, fading them in and out over 0.45 seconds.

In blind comparisons the fixed motion won on all three characters tested, with Celeste going from 1 to 3.5.

Why the arms stuck out

The walking characters held their arms out stiffly from their sides, which I described as a box shape, and it also showed up when they stood still.

Animation files assume a starting pose, called the rest pose, and Mixamo's rest pose is a T-pose with the arms straight out, while our characters were built in an A-pose with the arms angled down.

Copying the motion without correcting for that difference kept the arms lifted, so the agent corrected every bone from one rest pose to the other, and now the arms hang naturally and swing opposite the legs with bent elbows.

It also found that breathing motion was slowly building up in the shoulders and could drift an arm over a character's head, and fixed that too.

Seated characters keep still

Even after that, seated characters bobbed slightly, so the seated pose is now one held pose with the hips and spine locked.

Breathing, small head movements, blinks and eye movement come from the face layer only, and when a seated character talks only the arms gesture, at 30% strength, with no bouncing in the torso or shoulders.

Walking, the camera and the opening

The walk is slowed to a real walking pace of at least 1.35 metres per second, with each foot locked to the floor while it is on the ground so characters stop skating sideways.

Briggs also used to enter sideways, which I rejected, so he now walks straight through the door and opens it by hand.

Earlier versions had the dialogue camera push in towards whoever was talking, after the art director asked for closer framing, and in the final motion pass the push-ins were removed across the game, so the camera now holds with only a tiny drift.

The opening was retimed too, so the door opens, Briggs walks out, stops and faces you, and only then speaks his first line, which was a new Drama 3 take made with one API call.

Briggs's new opening line
[warm, steady] Hi. Whatever you need tonight, Agent, you ask me.
Briggs's new opening line
[warm, steady] Hi. Whatever you need tonight, Agent, you ask me.

The twelve versions

The first version came straight from my one goal, and after that every version came from me playing the game and writing down everything that bothered me.

Even Opus 5.5 can't help adding AI tells to an interface, so most of my notes on the early versions were some form of "simplify this".

VersionWhat I saidWhat changed
1Way too many words on screen, and the rooms look like AI image generationThe first interview had 147 words on screen and the accusation form had 304
2I'm overwhelmedEvery screen shows one thing at a time, and a test fails any normal screen over 20 words
3I don't know who anyone is or what the buttons doSeven short cards explain the situation first, and three AI testers went from two of five right to five of five
4It's just not real enoughEvery character became a full 3D person in a 3D dining car
5The art director gave it 5.5 out of 10Better staging, Crane's whole body in the room, and hands, hair and mouths rebuilt
6Eighteen notes, starting with "Who is Desmond Crane?"A new opening, four plain questions, and a corkboard notebook
7The room still tells me where to lookOne goal, "Search Crane's room", so you find clues in any order
8The live room isn't as good as the Blender renderShine, haze and moonlight took the room from 60% to 80% of the render
9I want the 3D as realistic as possibleFour agents worked overnight, nine of my thirteen notes passed, and 120 card titles were rewritten in plain words
10Too many long pausesSilence trimmed across 775 takes, and every room opens straight into 3D
11Make it scarierCrane's radio show plays from the dead, the lights cut out, and a figure walks past the windows
12Fix the claymation look across the boardEvery character rebuilt on MPFB with real motion capture

The notebook is a good example of how much the interface changed, because it started as a dense panel of tabs and ended up as a detective's corkboard you can drag and zoom, with cards that flip over to show what each person has told you.

The interviews went the same way, from a screen with topics, evidence panels, a hold-to-speak microphone and drag-a-card-onto-a-statement, to four plain questions with the suggested one first.

Not every idea survived, because version 11 also flashed a face at the window behind a suspect, and when I played it I thought it looked terrible, so it was cut.

After version 12 came the two days of character work described above, with the photo heads, strand hair, the new renderer and the motion fixes, and each change went live only after it won a blind comparison against what was already in the game.

The making-of site has a before-and-after slider for every one of these notes, and the two endings are at the endings.

What went wrong, and what to do about it

These are the problems you will most likely hit if you build something similar, with what fixed each one.

Too much on screen

Cap the words with a test. Opus 5.5's first interface put 147 words on the interview screen, and asking it to simplify only worked for good once a test failed any screen over 20 words.

Agents grade their own work too kindly

Use blind reviews. The agent building a character scored it about 1.5 points higher than a fresh reviewer did on the same frames, so every change shipped only after a fresh Claude compared shuffled, unlabelled pairs with the answer key kept somewhere else.

Small crops hide problems

Check full frames and video. One agent reported Nora's walk was fixed after looking at small crops, and when I watched it her arms were ballooning and her hands were curled up like claws.

Test tools can lie

Check the capture before the character. A character that seemed to have its eyes stuck shut was really the screenshot tool freezing in the middle of a blink, and the same file captured twice showed shut eyes once and open eyes the next time.

One shader change can break everything

Re-capture after every shader merge. A merge conflict in the skin shader once used a variable that only existed in another branch, the shader failed to compile and every face in the game vanished.

Your own machine is a shared resource

Block heavy work while you record. An agent ran a three-minute test while OBS was recording and caused skipped frames, so the agents now check OBS and wait for a gap, or use a rented GPU instead.

Prepaid balances run out quietly

Keep an eye on credit. RunPod ran dry in the middle of a session and silently killed a running machine, and another image service's credit ran out overnight, so the agents switched to a different provider.

More characters cost more of everything

Start with fewer. I made nine characters plus the detective, and every rebuild, review and fix had to happen for each of them, when I probably could have shown the same thing with two.

Plain words beat clever ones

Rewrite jargon. The story agents loved phrases like "milepost 212" and "the Box 1129 letter", so version 9 rewrote 120 card titles, every hover label and 70 spoken lines in plain words, and "the Box 1129 letter" became "the blackmail letter".

Fish Audio Drama 3

Ad disclosure: Fish Audio sponsored this video and added credit to my API balance. Everything I've written about how the voices were made comes from the build.

Drama 3 is the model behind every voice in this game, and it is what let the voice director treat the cast like real actors, with an acting direction on every line, one-take conversations and a timestamp for every word.

Fish Audio's own figures put it at number one on Artificial Analysis's pronunciation robustness test, reading 93.2% of hard words, numbers and codes correctly, and its weights are open, so you can run it yourself.

The directions work in more than 83 languages, and it also supports multi-speaker dialogue that keeps each voice the same and phoneme control for exact pronunciations.

Voicing this whole game cost about $1.40, and until November 12 the Drama 3 API is $10 per million characters, which is 80% off the $50 list price.

If you're building anything with voices, from a voice assistant to a game with characters you can talk to, I'd recommend trying Fish Audio Drama 3 through the API, and I want to try it next for characters that talk back in real time.

Try Fish Audio Drama 3 (API)

Fish Audio's newest voice model, with Voice Design, acting directions, multi-speaker scenes and word timestamps. This is my affiliate link.

Open

Build your own

Everything in this post, from the agents and their review rules to the Blender scripts and the lessons from the characters, is in the Game Studio plugin, and it comes with a playable murder mystery you can try in your terminal.

You can play Dead Air on the Meridian in your browser, and the plugin is on GitHub at per-simmons/claude-game-studio.

What did you think of this post?

Comments

Sign in to join the conversation.

You might also like.

Articles, builds, skills and tools on the same idea.
More At Work

AI FOR MORTALS.

Your daily AI edge that will make everyone wonder how you do it all.

Every weekday, the practical AI updates that matter, and exactly what to do with them.

The latest in AI, in plain English. Free, every weekday.