Stop motion video game Harold Halibut is Wes Anderson meets Monkey Island - here's why its "artists first production values" matter

Harold Halibut is not your usual video game. While Triple-A devs push for graphical supremacy using Unreal Engine 5, German indie studio Slow Bros. turned to stop motion animation and an 'artist first' workflow that celebrates the creative process as much as it strives to find a new way to deliver interaction.

I have the chance to interview Harold Halibut art director and studio co-founder Ole Tillmann to discover how and why this small team decided to spend almost a decade making an elegant animated game that resembles Wes Anderson filtered through classic era LucasArts adventures like The Secret of Monkey Island and Maniac Mansion. In a word, it's unique.

Harold Halibut features hand-made clay poseable puppets and detailed model sets, which are then scanned and digitised into Unity, with some clever use of mocap suits from Xsens to bring the scenes to life. This is a very modern and inventive merging of traditional animation and leading edge game technology, one that celebrates the flaws and quirks of artisan animation as well as video game narrative design.

Below you can find my questions and Tillmann's answers, I'd made some small edits to both questions and responses to ensure better readability. Enjoy this dive into the creation of one of 2024's most original games.

Did you ever feel like making this game 'traditionally' in 3D software?

'We started even longer ago, I fear to admit, and I’ll happily list some of what we’ve learned since, but also feel like prefacing that a lot of our time and learning revolved around running a game company, finding money and all the boring things, as well as actually realising and, in part, inventing a physical-thing-to-interactive-virtual-thing-pipeline. We never felt like doing this ‘traditionally’ in 3D, although we’ve often drawn comparisons out of curiosity.
"The very first reason for doing it the way we did it was that the first three people working on the game had no other way. Building things was their strongest way of telling a story, there wasn’t enough digital 3D knowledge or even 2D drawing at the time.

"I came on to the project shortly after and the building-then-digitally-rebuilding, manipulating and expanding was part of the big intrigue to me. I was working as an illustrator at the time and always connected to this way of thinking, some sort of hand connection first-digital processing later; pencil sketch to digital finish of a drawing is a simple example.
"We then found a really vivid connection as a group over the world building. Again that was informed by sitting there with actual little rooms in front of us and wondering what story to tell within them. Also everyone here is driven by experimentation and curiosity and this was a big way to connect all the different people and their expertises in really different areas over one very specific thing to somehow realise."

The game is incredibly ambitious. Has it grown over time, is this the story you always intended?

"It’s the story we always intended but I think it became a lot more fine grained over time. It’s kind of crazy how much extra detail one can forever fit in between going from A to B. It’s an 'artists first' production and never all too 'money sane', so we really enjoyed going all out on fine tuning this world and its inhabitants."

What has inspired the animation of the game?

"Stop motion films and films in general. In Germany we had a lot of the Czech classics of animation on TV, growing up. Also weird local / other European stuff and eventually shows like Celebrity Deathmatch on MTV. Also Aardman in theatres, eventually Wes Anderson. I’m sure all that had an influence.
"Even though we also grew up through the proliferation of 3D animation in pop culture I’m pretty sure the whole team shares a closer emotional tie to the practical effects and sets from the things we watched. But yet we all also share a nerdy childhood of picking up on random software at really early ages - film cutting, animation and 3D - we all had our first significant home-movie-making experiences digitally."

How are the characters and scenes made?

"Characters all roughly went from pencil sketches, to polymer clay puppets with sewn clothes, 3D scanned via a camera, two lights and a turntable, retopologized, rigged, animated largely via mo-cap and some more specialised hand animation.
"Sets went from pencil sketch to digital illustration, made of all kinds of different materials, a lot of wood, metal, plaster, painted, 3D scanned in individual parts and reassembled and lit in 3D.
"Software was all pretty normal major-brand-industry stuff. Most of the specialised tasks went through all kinds of iterations. 3D scanning ended up being mostly from Capturing Reality, which does photogrammetry. We had a lot of help scanning the environments (flat surfaces in particular) from a befriended engineering business that were working on an incredible 3D scanner which has since disbanded, sadly.
"So that was all custom hardware in particular which they were building and assembling themselves. We really just used whatever we could to accentuate all the things we wanted to show through from having that analog foundation. Unity is our game engine."

Have you developed your own approaches or even your own apps and tools for the game?

"Yeah, we have developed our own approach for sure. Biggest was figuring out how to go from the things we built to 3D objects that could be handled in a game environment (it has to be really efficient and lightweight, usually.)
"Then we did a ton of tweaking to land on the right spot between expectations for traditional stop motion animation and game animation / feel. Frame rate is a good example. So games usually work at varying frame rates and some people are really used to having that 60 fps interpolated, milky animation look, while stop motion tends to work between 12 and 24 fps and relies a lot on our brains connecting the frames.

"Since it's done by hand and not calculated by a machine there are little imperfections that are part of the technology. We focused on how the game felt play-wise first and then adjusted animation, light, scale etc to convey that sense of realness that stop-motion gives off.
"Plenty of custom tools were made, particularly for animation and repeat processes like wrangling all the mo-cap data and how to reliably place and sync all the little parts in a scene (characters, their motion baths, objects they interact with, lights, etc.) A lot of that was Ilja Burzev’s work, who’s our brilliant 3D technologist.
"Our composer (and director and lots of other titles) Onat Hekimoglu rigged up some insane audio chains for the soundtrack. My favourite was a group of wooden robotic puppets that he controlled through a chain of keyboards / synthesisers that would sing together (they sound like Vocaloids so think Daft-Punky robot voices) into a microphone that in itself was made to mimic other historic microphones. He then had other virtual voices harmonise with the robot choir which he then fed into another piece of software to produce music to."

Do you use mocap? Have you made any cool gadgets to help development?

"Yes, we used mostly mocap and one rig is our Xsens gloves. We tried a bunch of different suits, gloves, etc and the biggest hurdle for us was not having a big studio space to record in.
"We used mocap because of the big story but relatively small team we had to make it all happen. And because we quickly realised that it works pretty well. The puppet element makes for a good balancing element when it comes to the sometimes awkward quality that stems from mocap animation.
"The only cool gadgets we made were extremely low-fi reference tools, like stacked boxes for different heights to interact with, scribbling portraits on printer paper and taping them to sticks, jumping onto couches, jumping off of chairs, leaving bedrooms covered in tape for all the reference points, stuff like that."

What does the handmade process add to the game and how it feels to play?

"Hand-building everything I believe makes for a unique feeling while playing the game. It’s hard to describe and I don’t know that most people could point to it, I’m the wrong person to say, but we quickly understood from the beginning that building the game physically first deepened the immersion into the made-up world.
"It’s a little bit the opposite direction of what the photorealism chase of a lot of games does. You know as a player at first sight that this has been created or came from someone's fantasy and its not a real place but then the hand-built part really helps bridging that gap and gives you some subtle conviction that you are deep inside of a reality."

Does the use of clay and hand animation give the narrative and gameplay a new feeling?

"I hope so, it does to me. Narrative, in that we came to the story via iterations that were based on the building from real materials and looking at them together, writing more, building, etc. Gameplay is hard to say, but to me it has something to do with the relation to puppeteering and people’s experiences as children with puppet theatres and yelling at the box with the puppets inside about what to do or who to look out for.
"I think working by hand in combination with contemporary storytelling methods like gaming and virtual building could also be compared to the history of film and TV production. There’s definitely a public realisation there, too, that ran pretty parallel to our long production timeline, where people are going back to practical, in-camera effects and worlds. Building sets and puppets for actors to walk through and interact with rather than the humungous green halls and tennis-ball-creatures and people in green suits, etc. that were then replaced by completely digital worlds after filming.
"I think there was just a spike of excitement for digital creativity for a while and now its readjusting and landing on a balance between what was learned in the past about telling stories and the expansion of possibilities since."

Why did you decide to hand animate, and have you added in 3D animation to compliment it?

"Yes, while we didn’t do a lot of hand animating the hand building of the game’s world and characters and the interweaving with 3D building and replication was a fundamental thing for the game’s story, look and feel. Because we didn’t know how else to do it at first, because it was a lot of fun to make this way and because it was an interesting challenge to pick up on an established but somewhat historic way of doing this (namely photographic stop motion animation) and coupling it with contemporary, interactive and experimental ways of storytelling.
"We think there’s a lot that works well when it comes to visualising made-up worlds by building objects. And while virtually building made-up worlds works well in itself we were just really curious about how to use what we liked about physically built worlds together with the contemporary interactive world.
"I think it’s really important for us to emphasise that this wasn’t just a retro-for-the-sake-of-it move. Not forcing the stop-motion into a game context but feeling out what we like and want from both and try to combine them."

What do you use for VFX in the game?

"Lots of little tools and plugins, Unity connects to all kinds of third-party software. It’s mostly what we were able to find that fits into that stop-motion-built way of thinking and is still cost-effective when it comes to having a game for all the platforms and their limitations.
"So, for example, we had to unfortunately drop our water surfaces that we used for a while that were looping through different frames of 3D scanned cling-wrap foil. This looked really nice and fit so well into the world but just wasn’t feasible to use in the end.

"Also we coupled a plant-distribution software with our plants that were made from cut-out watercolour drawings. So they were all built and 3D-replicated, but then a specific program had to copy them across the terrains because, again, this was the only way to do it within the restraints of machines.
"Also there’s a particle system that just runs through loops of these tiny plankton watercolour drawings. We also used painted textures on a lot of, if not all, transparent surfaces and lamps in the game. So picture a white clay light bulb with a light air brush layer of yellow on it and then a 'dirt-wash' of a dark colour that shows all the dents and textures of the clay by sticking to the crevasses. That then is programmed to digitally glow and that looks different from having a digital object with 'light yellow' assigned to the hex-colour code give off light."

Why choose Unity?

"We chose Unity because it was very accessible and could easily be extended thanks to the huge community and third party devs. I think the biggest game changer for us was HDRP (High Definition Render Pipeline) being introduced and we’ve since connected with the Unity devs on many occasions to talk about camera and film related things and how to tie them into game dev."

Is animation in a good place?

"I don’t actually think I’m qualified to answer that question, since I don’t feel I have a particularly broad understanding of animation's place in the world. I mean in many ways, especially in global economic ways, it surely isn’t, but in terms of experimentation and open mindedness, yeah!
"Two big things that come to mind are of all things Disney buying the Star Wars franchise and embracing all the physical world building and Scavengers Reign, a recent 2D animation show on HBO. Both because it always needs a money management person to have a simple reference for where to put money and I think Disney is a good hallmark to point to and Scavengers Reign is just an incredibly well made and beautiful show which seems like its also successful from a spreadsheet perspective (at least I really hope so).
"While I think it’s completely digital, the 'hand-made' quality is in the animation and people really imagining motion in their minds and putting it to figurative paper. Also the attention to detail, there are so many micro expressions, specific ways of touching objects and other emotional hooks. I loved it!"

What was the hardest part when it came to digitising your handmade models?

"We really made a point to try and somehow translate all the things we built, digitally. A challenge was glass and reflective surfaces in general, because they’re still a challenge to scan. But then we’d find interesting work arounds like the glass thing I mentioned earlier with the light bulb.
"We also really enjoyed eventually playing with how much digital-only stuff we could add like fog or lights or effects that would still fit into the world and add to it from the digital side."

Is the barrier between games and filmmaking getting closer, both creatively and in terms of the art tools and software?

"Yeah, I think so. Both industries are buying into each other (Unity famously bought Weta, for example) and Unreal Engine was used for that crazy 360 LED panel they used on the Mandalorian a bunch [a LED Volume set). If you haven’t heard about it, it’s basically a green screen but it just shows what's in the background instead of being added later. Plus it's tied to the camera, like the physical camera was tied to the virtual background like a 3D camera would be and they played with that. They would still build a huge aircraft from physical materials to put in the middle ground a lot, interestingly.
"So film is picking up on game dev and generally both just have digital work as a common ground. But also games have been inching in on film work, but a bit wonkily so. Looking back, games just weren’t really related and, admirably so, had a completely unique language going on.
"Then I think there was a slightly awkward phase of mimicking film and getting that cinematic quality going but so many times without the actual film experts on the teams. And by now I think it's a nice mixture of many film people working on games and vice versa and yes it's all melting together."

What are the plans once the game releases? Can we expect a TV series or movie?

[Laughs] "Please don’t expect it, no. We made this a game on purpose. I think the way we made it its so much about existing in this little world and finding the people inside it on your own, at your pace.
"Surely it could be adapted and developed for TV and movies but there are no concrete plans so far. We did however just spend ages developing this game-making pipeline so we’ll definitely be using that in the future.
"I don’t think I’m allowed to announce any of the other things planned around the Harold story yet. But we’re working with a few different companies on additional things."