This story is over 5 years old.

Creating Movies that React at the New Cinema Hackathon

Merging the souls of gaming and film at last.

Originally published at The Creator's Project.

Cinema as a reactive experience is what’s explored in the latest installation from one of the teams at our New Cinema hackathon, which took place at the end of last year—a collaboration between The Creators Project, Eyebeam, and Framestore. Together, members of the creative coding community and the award-winning VFX studio explored how filmic experiences might change in the future in this installation inspired by the ancient past.


Pooling their knowledge of programming languages and CGI skills, the team behind this latest experiment took their inspiration from what were arguably the earliest known instances of visual storytelling: paleolithic cave paintings. The team—Eyebeam residents Nick-Fox Gieg and Ramsey Nasser, Google coders Alex Kauffmann and Boris Smus, and Framestore visual masterminds David Mellor and Mike Woods—created a game/cinema mashup called Before the Flood, a dark cave environment that can be explored by people walking around in front of a screen, created using the Unity game engine and floor-tracking software developed at Google.

The team call it a “‘slice of cinema,’ a proof of concept for new avenues of exploration,” and it uses Kinect cameras to track the participants, who can stroll around the cave by walking around in a designated area in front of the screen. As each new person enters the installation, they effectively become a new torch that sheds light into the darkness. This way, participants move the camera collectively, while shining torches individually to reveal cave paintings that tell a story. In this way, it maintains the shared experience of going to the cinema while also incorporating the individual exploration of a video game.

The details of the cave were created by hand drawing image maps in Houdini and combining these models with After Effects animations and Kinect motion capture, before processing it through Unity. Creating a cinematic experience using a game engine isn’t without its challenges, and coming from a background in cinema VFX, Mike Woods from Framestore expresses the concern of merging the two. “The word ‘game engine’ is a bit of a red herring for us and it’s all about, can you get that level of quality people might expect from the cinema into a game engine environment? It’s all about the texture qualities, the lighting, all of the things that make something look photoreal. There’s a lot of brilliant console games out there, but they still look remarkably different to feature films.”


And this is where the artistry and ingenuity of the coders comes in, translating their skills over to the high-powered tech that Framestore has been using for many years. It’s this cross-pollination that holds the real potential for the future of cinema. As Woods remarks, it’s about “Blurring that line between what’s experiential and something you can sit back and enjoy,” while also maintaing the cinematic quality that people have come to expect.

You can check out the previous team’s effort We Make The Weather below

Follow the teams’ progress at