* Amendment [3/22/22]: Originally the author referenced Superman (1978) as the first use of 5.1 audio, but it seems that the first instance was actually Batman Returns (1992).

Wind swirling around you, rain dripping behind you, a plane taking off right above your head- these are some of the illusory possibilities that immersive audio systems can produce, specifically object-based audio systems. In this sound mixing method, audio clips can be treated as objects and moved anywhere in a 360° space, as opposed to being sent to a speaker in a fixed position. This works through algorithms that divide the sound between several directional speakers. As the speaker outputs intersect, the sound will appear to come from the assigned coordinate. After the mixing stage, the sound object coordinates can translate to whatever speaker system is available, from headphones to movie theaters. The sounds will occur in the accurate relative position, though the more speakers that are available, the better the reproduction will be. Our hearing works in three dimensions, so we can perceive whether a sound is coming from front or behind, left or right, and above or below. In theaters with a multitude of speakers on the walls and ceilings, this natural perception can be replicated with technology, enhancing the already immersive experience of seeing a movie.

Surround sound for film began with just two speakers, placed on either side of the screen. The next upgrade was LCR, or Left Center Right, which added a speaker behind the screen. This allowed audio to be sent to three separate channels which allowed the soundtrack to be more spatially accurate, and more sounds could occur at the same time. Three-One systems, started by 20th Century Fox in the 1950s, used LCR plus one surround/effects channel, which could be played from as many speakers as available. These surround speakers were positioned evenly around the rear of the theater in order to reach the full audience. This provided a way for sound to come from other parts of the room, not just the front wall. Various other multi-speaker setups were tried as the film industry grew, but not until 1978 did a format emerge that we still use today, 5.1 surround sound for the premiere of Superman. This system consists of five channels, Left, Center, Right, Back Left, and Back Right, plus a subwoofer. A subwoofer is specifically made to produce bass frequencies, which can be used in film to simulate explosions, earthquakes, and other large impact events. Low frequencies have very long wavelengths, meaning that we’re less able to perceive their source, so their position in a room won’t affect the listening experience much (thus the designation of .1). Today, mixing with object-based audio systems is increasingly common, enabling sound effects to appear in the exact space they exist on the screen, and into the theater environment.

The two companies with object-based audio theater systems currently are Dolby, with its Atmos system, and DTS:X by DTS. Dolby Atmos can handle up to 128 audio tracks at a time- a 9.1 bed and up to 118 audio objects. DTS:X can support an unlimited number of objects at a time. Atmos and DTS:X both include ceiling or height speakers which make it possible for sounds to appear to fall down on or pass above you. In theaters with enough speakers, every seat offers a captivating audio experience. There are other movie theater systems that include height speakers, but don’t use object-based audio rendering, like Auro 3D, and NTS 22.2.

When making a project with object-based audio, some sound mixing practices remain the same. Dialogue nearly always comes from the screen, rather, from the speakers behind the screen. Not all sounds need to be objects either. Light stereo ambiences and room tones usually don’t need to be mixed to move around a space, they can be a bed for other active sounds. But for these active sounds, the creative possibilities are wide. Object-based audio creates a combined opportunity for accomplishing artistic intent, and enhancing the audience’s experience. The first film to be released in Dolby Atmos was Disney/Pixar’s Brave (2012). With an adventurous plot and playful animation, this movie set the tone for the types of production content that use immersive audio mixing. Dolby Atmos releases have continually increased since 2012, but the genres of Action and Kids/Family remain the primary movie types. Action films usually make up about 50% of productions using Atmos per year. Out of the 38 Dolby Atmos theatrical releases set for 2021, (a much-reduced output due to pandemic-halted productions) 40% are action films. DTS:X productions follow the same trends. Over time there’s been gradually increasing output of object-based audio documentaries, comedies, non-action narratives, and musicals, including In The Heights and West Side Story, both coming out this year.

One example of a movie in the Family genre is The Missing Link (2019), which used object-based mixing very playfully. In the beginning of the film, as a wave crashed over monster-seeking adventurers on a lake, it also crashed over the heads of the audience. When the monster roared, a huge bass rumble was sent through the theater. Later when a character said “Hello” into a cave, the echo bounced exaggeratedly from one side of the room to the other. In a fight scene, the sounds of glass breaking and shouting came from the far back corners, and gunshots traveled fully across the room. One of the few times dialogue didn’t come from the screen, it came from the back wall, when one character approached another from behind. Another scene used object-based audio capability to emphasize the romantic feelings between two characters. As they walked through an Indian jungle, the sounds of critters and music floated around the room to create a magical sensation. Every musical track swelled and traveled above, behind, and in front of the audience. The Height speakers were essential to create this effect.

Another film from the Action genre, Avengers: Endgame (2019) used object-based audio less comically, and more for battle impacts. There were moments in the film where characters traveled through a time warp accompanied by an immersive bass rumble, which made me feel like I was flying as well. In another scene with a mushroom cloud explosion, a powerful bass wave was sent through the theater. There was another distinct moment when the main villain, Thanos, threw his massive sword like a boomerang, the sound following as it swirled behind the audience’s head and back into his hand. In the fight scenes, all the little elements were carefully panned to really fill the image. However, the music often took over opportunities to create more immersion. In the fight scene in The Missing Link, the brawl spread to the far corners of the theater, but in Endgame the sound effects only covered the direct action on screen. Though both are fictional genres, Action films are often made to seem somewhat realistic, while Family animations can exist in pure fantasy.

When theaters finally opened this May, I saw another film in Dolby Cinema, a re-release of Scott Pilgrim vs. The World. My favorite audio moments were those I could follow as they traveled across the room. For example, a ringing school bell sound object accompanied the onomatopoeia text “BRRRRRIINNNGGGG” as it flew off the screen into the left corner. Another time when a character roller skated through snow, making it fly off the right of the screen, the audio followed out to the right wall with comedic timing. In one of several musical battle scenes, piles of plastic cups were swept into a cyclone, with the small, satisfying sounds of empty plastic cup collisions spinning around the theater in large counter-clockwise circles. In another musical battle, an EDM performance created a tidal wave of sound that shot through the audience, causing destruction on screen, and hitting the theater seat at the perfect time. As a movie filled with sonic jokes, video game references, and lots and lots of music, this was a treat to experience immersively. For now these releases are still special viewing events, but as the possibilities of 3D audio storytelling become more popular with studios and directors of all genres, more theaters will incorporate Dolby Atmos and DTS:X technologies. If you’re considering a return to the movie theater, I would recommend seeking out an immersive audio experience.