How Technology Enables Immersive
Film and TV Experiences

Have you ever gotten lost in a book and completely lost track of time? Great stories tend to do that. They immerse the reader. Reality fades away, and it’s as if you’re part of the story itself.

And as we all know, a great movie or television series can do the exact same thing.

There’s something different about on-screen storytelling, though. When you’re reading a book, you create images of the story in your mind. You envision how the characters and settings appear. Movies and TV aren’t like that. Instead, a director and the actors have control over the characters, their movements on screen, and the visual and aural elements you encounter (or don’t).

If you’re familiar with the concept of mise en scène, you know that everything on the screen or the stage is there for a reason.

A staircase in the background gives characters a way to escape. A cigarette between an antagonist’s fingers signifies power. A flickering light adds tension and suspense. They all play a role in the story.

Unlike, say… a novel, a film must rely on mise en scène because directors have only a few seconds to show viewers where the action takes place. Authors have unlimited space to set the scene. Directors don’t, but by creating mise en scène, they actually have more control than an author over how a scene looks and feels. Every shot, every angle, every sound and piece of sound is there for a reason. To help bring the story to life.

It’s up to the director – not the viewer – to interpret the narrative in its original form, portray that interpretation on screen, and create an immersive, sensory experience.

While authors rely on imagination, directors can tap into two of the five primary senses -- sight and hearing.

A Tale as Old as Time

Emerging technologies like virtual reality or new photo-realistic gaming engines make it clear that we are moving towards entertainment experiences that simulate reality. The closer we can get to making someone feel like they’re experiencing it themselves, the more “real” it feels.

Over time, media creators have continually expanded their works’ immersive power by exerting increasingly greater control over a story’s portrayal.

Before the Epic of Gilgamesh, storytellers relied on oral storytelling. While this has its own romantic merits, the story was (of course), different every time. Each telling was unique to the audience and the storyteller and it was likely difficult to retain a deep level of detail over time.

Gutenberg’s printing press enables creators to mass-produce written narrative and preserve the story for readers. While each reader could imagine characters and scenes differently, writers had a level of control over the information that informed the readers' interpretations.

By the 1920s, radio offered a new way to experience narrative. Instead of asking you to imagine a scene in your head, producers used music, sound effects, and oral delivery to heighten a story’s immersive qualities. In other words, they made the experience more immersive by controlling more of it.

Around the same time, cinema was gaining a foothold in popular culture. Not only could creators use sound to immerse their viewers – they could manage the entire visual experience! With seemingly limitless ability to frame each scene, film offered even deeper immersion opportunities than theater.

Stereophonic sound eventually became the norm in the film world, debuting for large audiences in 1953 with This is Cinerama and House of Wax. Using stereo sound, filmmakers could further engage viewers and bring them into the experience.

Technicolor, first developed in 1915, also matured considerably by the 1950s, making films more visually compelling and true-to-life.

And so it goes. With the advent of each new media technology, creators gain more control over the reader/listener/viewer experience. The result isn’t greater immersion – a good book might immerse you more than a bad film – but a multi-dimensional immersion in which all sorts of experiences are possible.

While most of the technologies start in commercial cinema’s, today, home theater technology doesn’t lag far behind. The average consumer has access to home theater equipment that allows them to reproduce these experiences in their home, whether in a custom home theater room, a media room, or a family room.

Improved video technology promises immersive potential.

If you’re an adult today, you’ve experienced an incredible increase in video resolution.

In the 1990s, 480i video resolution was standard. It’s what you saw when you popped in a VHS tape or flipped to the Arsenio Hall Show. Since then, we’ve come very far. DVD used 480p. Soon after, when TV screens got bigger, neither 480i or 480p made sense anymore. We started buying HD televisions, first at 780p and eventually at 1080p. Not only could the 1080p resolution properly fill our larger screens – it looked a lot better, too.

Now we’ve got 4K televisions, which offer a whopping four-fold resolution increase over a typical television purchased in 2012! Modern TVs and projectors can also produce many more colors and variations today than ever before and have better contrast ratios.

Refresh rates have also improved. If you’ve never seen a 240Hz or even a 120Hz TV, it can be a strange experience at first. People often describe that until you get used to it, it feels like looking through a window or watching the actors live on set. Once you get over that initial reaction, it just feels more real!

Just as Technicolor made movies more true to life, improving video technology makes the on-screen narrative even more vivid. It captures your attention in a way that your old VHS tapes never could. Pretty soon, we’ll be watching 8K televisions. Who knows where it ends.

Audio technology also brings us into the story.

The same trajectory holds true for audio. As technology improves sound quality, listeners enjoy a richer experience. And in the world of home theater, audio technology is expanding film and television’s immersive potential by leaps and bounds.

This is nothing new, of course. Starting in the 1970s, Dolby Stereo brought four-channel audio into movie theaters. By the 1980s, you could even enjoy Dolby in your home using a Dolby Pro Logic decoder.

In a sense, Dolby technology created modern home theater by making it possible to enjoy true, cinema-like audio inside your living room. Dolby Digital, introduced in the 1990s, further pushed the limits by giving each speaker its own audio track. As a result, sound engineers could direct sounds to different speakers however they wanted to!

Which brings us to today.

Object-based audio systems Dolby Atmos and DTS:X are able to create effects that convince the viewer that a fly is buzzing in circles around their head, or that someone is breathing over their shoulder. The level of control that sound designers have today, to precisely place sounds in 3D space, creates a level of immersion never before possible.

How much more immersed can we get?

In just the last two decades, we’ve gone from 4:3, 480i televisions and basic, two-channel audio to 4K TVs and Dolby Atmos. That’s quite a jump!

So, what can we expect in the years to come? 3D television has always been a niche technology, and many have declared it dead. But could it still take off? Probably not, but it’s anybody’s guess.

Maybe virtual reality will make inroads. Maybe it won’t.

Here’s a relatively safe prediction: We’ll keep seeing higher resolution video technologies. 8K television is already on deck and after that… who knows? Some critics claim that resolution increases have diminishing returns as the human eye has a finite ability to distinguish the improvements. That being the case, at what point can do we declare screen resolution “mature?”

Could technology eventually lead to even more immersive experiences, tapping into more of the five senses? Smell-O-Vision may be a long-running joke in the entertainment industry, but who’s to say where the future may lead!

One thing is for certain. The “greater creative control + technology = immersion opportunities” formula will continue to deliver exciting experiences. That’s what it did throughout the 20th century. There’s no reason for it to stop now.