Justin Gray is an award-winning producer, mixing engineer, and mastering engineer based in Toronto, Canada. Justin is a leader in the field of immersive audio mixing and mastering, where he works with artists, producers, and labels worldwide to bring their music to life in the Dolby Atmos and Sony 360 Reality Audio formats. His studio, Justin Gray Sound, was one of the first worldwide to be certified for immersive music mixing.
Throughout his career, Justin has enjoyed working with a diverse range of musical styles. Some of his recent clients include Olivia Rodrigo, Snoop Dogg, Brandy, The Tragically Hip, Tiësto, Valley, Arkells, Mother Mother, The Sheepdogs, Lola Brooke, Josh Ross, Jamie Fine and more.
I recently had the chance to chat with Justin about his immersive audio journey, the complexities of remixing classic songs in Atmos, and where he thinks the industry is headed.
Tell me a bit about your background. What made you choose to pursue a career in audio engineering?
My journey in music actually starts as a performer. I attended Humber College–where I’m sitting right now because I just finished teaching–as a bass student over 20 years ago. Since then, I’ve had a pretty long career as a professional bassist, touring and recording with folks not just here in Toronto but all around the world.
Even going back to my late teens, I was always fond of the recording arts. Whenever I had the chance to play bass at a studio session, I was the first one to walk up to the mixer or recording engineer and ask questions.
Then, as soon as I graduated, I found myself developing my Pro Tools skills and taking on more production work, like editing records I’d played on and eventually mixing them, too. So, for about a decade in Toronto, I was quite regularly producing, editing, and mixing records. As time passed, my studio grew.
Eventually, I began to focus specifically on mastering, as the whole world of finessing audio and dealing with the most minute details is really interesting to me. I started by mastering some of my own projects, which were well-received within my community here in Toronto. From that point, I transitioned into full-time mastering work.
How did you get interested in immersive audio?
I’ve been listening to music in surround for well over 15 years now. I’ve got a good Super Audio CD and DVD-Audio collection, along with some quad discs that Jim Anderson made me aware of more recently. Some of my favorites would be those Lord of The Rings box sets that have 5.1 mixes of all the Howard Shore scores.
While listening to the discs on the 5.1 setup in my studio, I would always be thinking in the back of my mind, “ I wish I could be involved in the surround-world in some capacity.” A lot of folks encouraged me to go into the film side, but–even though I love film–I’m a music producer and wanted to stay in that era.
Around 2015, I started to hear the first whispers of Atmos for music. This was mainly from attending conferences like AES and NAMM, where I’d see people like Morten Lindberg, Jim Anderson, Ulrike Schwarz, George Massenburg, Leslie Ann Jones, Elliot Scheiner, Ronald Prent, Darcy Proper, Michael Romanowski, Gavin Lurssen and Reuben Cohen in attendance. The Auro-3D folks had an early presence at these types of industry events as well, so the immersive community was already forming at that point.
Personally, I’m very moved by listening in surround. I want to create music that way, both as an artist and an engineer. So as soon as I heard that Dolby wanted to get involved in the music space with this new format, I thought it made perfect sense. It seemed like an inevitability, so I started building an Atmos studio very early on.
It became a bit of an obsession, to the point where I had a full mastering-level 7.1.4 set up by early 2017. All matched Lipinski monitors, which was no small investment. My family and colleagues here in Toronto looked at me like I was crazy. [laughs]
So I started off by remixing records that I’d produced and my own music, even though no one was asking for it yet. I wasn’t part of that first group of Capitol Studios engineers that were working on catalog for UMG (who are some of my favorite engineers!), so I did not even have an outlet to release my mixes (outside of Blu-ray). By 2017, I’d gotten close with Ceri Thomas at Dolby and was certified as one of the first Atmos-equipped studios for music in Canada. In those early days, there were only a handful of studios (outside of film soundstages) that were equipped.
Then, just before the pandemic hit in 2020, Tidal started streaming in Atmos. They still do a great job with it, but it didn’t really break into the mainstream. I had colleagues who were still skeptical, but I had faith that either Apple or Spotify would eventually take it on.
Within a month of Apple announcing support for spatial audio, I started to get calls from the big labels–Sony, Universal, and Warner. Since then, I've mixed and mastered thousands of tracks in the format, both frontline music and catalog. It’s been just an unbelievable shift in what I do and the range of music that I work on.
I want to first ask you about the mastering process, because I think there are a lot of misconceptions about what this actually entails. Back when the industry was all-analog in the 1960s and ‘70s, there were clear delineations between mixing and mastering. As I understand it, the recording engineer would print their mix to a two-track tape and that would be sent to a specialist like Doug Sax or Bob Ludwig for those critical final adjustments.
Today, because of the nature of digital recording, those lines are more blurred. The mixing engineer can essentially ‘master in the mix’ by processing the two-bus. It’s gotten to the point where there’s an entire industry built around mastering plugins, like iZotope’s Ozone or Universal Audio’s digital recreations of analog bus compressors. So, in your own words, what exactly is mastering and what is your typical process?
There’s a lot to unpack there. [laughs] The simplest definition I can give you of mastering would be that it’s the practice of finalizing music recordings for release. On a slightly deeper level, it is the art of listening and reacting, with the goal of appropriately refining audio to ensure translation to listeners.
To ‘finalize’ audio is simultaneously an instinctual and technical practice. So what does it mean to finalize something? Well, it's completely contextualized on where it starts. A mix could conceivably come to a mastering engineer at a level where sonically it is finalized, as it has already reached its peak sonic potential. Most commonly, I am fortunate to work on music that has been mixed at a very high level and requires relatively subtle refinement to reach its full potential.
As a mastering engineer, my job is to listen to the audio objectively and make decisions on what to do, or not do, based on my own aesthetic preferences–which are based on years of experience listening to and working on a wide range of music.
Michael Romanowski described it to me as a “state of mind.”
Yeah. The most important thing that one must achieve when mastering is having the correct mindset, which is the ability to step outside of the inner workings of a song or a production and be able to look at the whole picture not only on a song level but also on an album level and understand how to balance all of those elements.
I do think that we are pursuing translation in all of our work. We’re trying to make decisions that allow the artistic intent to best translate to the world. This could, for instance, mean fixing a sonic problem that a production team didn't catch.
A common example is low-end sounding huge in a mix, but actually, being too robust in relation to the rest of the spectrum. By going in and taming one specific area with EQ, we're able to maintain that same low-end presence without it feeling quite so muddy in the midrange. Another common practice might be subtly lifting the very top end with an equalizer to get that extra bit of clarity–it’s like that last scrubbing of a window to the point where it’s crystal clear. Mix engineers are not always working in a mastering-level room, where half a decibel (or less) can mean a world of difference.
To your point, I do think that people often incorrectly assume that mastering just means making things louder. We are definitely at a time in history where making something loud is the least challenging it's ever been. Digital limiters have provided people the ability to reach levels of loudness without distortion that were practically impossible for mastering engineers to achieve in the pure analog days.
These tools are quite exceptional, but I will say that the mastering engineer needs to make a conscious decision about what is the appropriate loudness. It's really not about making it louder. It's about deciding, in the context of a song or a record or even a genre, what is the appropriate loudness for this song, and how do I get there? Should I get there via EQ? Should I get there via compression? Should I get there via saturation? Should I get there at all? Unless a mix is really in bad shape–which can happen–I’m usually making very small gestures using all of these tools. The goal is never to change something just for the sake of changing it.
It’s also about getting fresh ears on a project. For example, I’m working on an album right now, which I’m sending to Michael Romanowski for mastering. Now I’m a mastering engineer myself, and I’m capable of working in both stereo and Atmos, but I've played on this record. I composed this record. I mixed this record. I believe that what I’ve delivered him is as perfect as I could ever make it, but at this point, there’s no way I can fully see the light through the trees. It's impossible. I’m too attached to what I have crafted to be able to be objective.
So maybe, in closing, the mastering engineer’s ultimate goal is to provide an objective voice and implement the tools with great care, so that a production reaches its full sonic potential.
As you well know, audiophiles have been complaining for years about new remasters of old records that sound harsher and less dynamic than the originals. There’s been so much backlash that some labels have started including what they call ‘flat transfers’ in their reissues of classic albums, which–as I understand it–means that no additional processing is being applied to the digitized analog master.
This approach is certainly preferable to brickwall limiting, but not without its own share of issues. If you were to compare some of these flat transfers to the original LP or CD editions that were professionally mastered, you might pick up on some very subtle EQ adjustments–like the examples you cited before–that really help bring the final product over the finish line.
Absolutely, these things do matter. As aesthetics and technology have evolved over time, so have our practices.
Let’s say that someone made a beautiful sculpture one thousand years ago. We generally like to leave those alone and let them slowly deteriorate over time. That’s part of the beauty of the art. If a team of people were to then re-polish that sculpture so it can last another thousand years, someone who looked at it the day before might say it’s lost some of its original charm. I totally get that sentiment.
When we’re talking about a legacy piece of art, changes to that art might never have been requested by anyone. That said–the owners of these files are always looking to keep them up-to-date, which can mean creating new versions that will connect with new listeners. I see the value of this as well as keeping the old versions available. Someone should have that ability to connect with a new version of the art, or they can always go back and stick with the one that they’ve always known.
Remastering only allows for so many changes to be made to the music, because you’re making adjustments to a finished mix. Equalization can be used to set certain elements back or forward in the stereo picture, but you’re not able to fully remove or add anything. The cake is already baked, so to speak.
On the other hand, spatial remixing requires a complete reconstruction from the original separated elements. This is such a precarious undertaking, because now you can actually add or remove things and completely change the relationships between the source components. So when it comes to doing these new immersive mixes of older albums, what is your philosophy and workflow?
My philosophy is to always first understand the intent of what I’m doing. Every time I take on a project of this nature, ideally, I’d love to sit down with the artist, producer, or whoever owns the master and ask them, “What do you want out of this?” Obviously, there are situations where I can’t ask that question directly for a variety of reasons, but I certainly ask it to myself and try to fill in the gaps.
To me, a spatial remix means there’s a defined stereo (or surround) master and then a defined set of source elements I’m to mix from–it could be tracks, stems, or some combination of both. Typically, I am not being asked to reimagine the song but rather to adapt it into an immersive presentation (Dolby Atmos). What is happening is a spatial re-orchestration of the individual parts to some capacity.
What is definitely not commonly requested with the majority of spatial remixes are any changes to the perspective of what that piece of art was. To me, those primary perspective pieces are tonal qualities and relative volume levels. As a producer and mixer, I know that those final adjustments to the mix are what we labor over the longest–like half a dB up on the guitar in the last chorus, or a small touch of 14k on a vocal to let it poke out just enough without being too sibilant.
“As a mastering engineer, my job is to listen to the audio objectively and make decisions on what to do, or not do, based on my own aesthetic preferences–which are based on years of experience listening to and working on a wide range of music.”
So when I’m doing a spatial remix, as a base-level philosophy I want to be able to maintain the essence of the song but also to bring it to life immersively in a way that's meaningful. That balance requires so many different skills. First, you need the instincts of a producer to understand what the essence of a song is. The producer’s role is really to get the song to tell a story, and they did all that work to make the stereo (or surround) version tell that story.
Then, it’s the job of a mixer. The tricky thing about spatial remixing is that the tonality of a certain part can change when it’s moved in space, because that’s how our ears work. So the job of the remixer is to decide, do I like that change? Is it okay if a certain instrument naturally sounds a bit darker as it goes behind me, or maybe a bit less present as it appears above? Or do I want to adjust that to be able to maintain something that I now feel is missing?
It also requires the mindset of an arranger, because we’re orchestrating the song in a new way. I’m deciding that I want the background vocals in one spot, the percussion in another, the solo positioned elsewhere, and so on. These are all very critical production decisions, even in a stereo context. Nobody’s just panning by accident; you can be sure it’s been talked about and labored over!
Finally, in situations where there’s not going to be a dedicated mastering process–which I am not advocating for, but will admit is still very common in the immersive world for all sorts of reasons–you better have the ears and capabilities of a mastering engineer because you're expected to deliver the master. That doesn’t just mean hitting the -18 integrated loudness target. Anyone can reach a target number so long as they understand the parameters of how it’s being measured. That is not mastering.
Mastering music in immersive audio is about getting the appropriate loudness combined with the appropriate overall spectral and spatial balance. Just because it’s not going through a two-bus doesn’t mean that the same skills that a stereo mastering engineer applies aren’t equally valid here. So I think, as a spatial remixer, that the discipline actually requires someone with an exceptional audio engineering skillset and range of experience.
Spatial audio can be incredibly revealing and thus sometimes unforgiving to older music recorded in the analog days. Certain imperfections in these classic records, like signal bleed between tracks or the occasional out-of-tune note, could be somewhat hidden in a stereo presentation. In a 12 or 16-channel environment, you risk overexposing these types of things. Going back to your analogy about polishing an old sculpture, it could be viewed as kind of sacrilegious to ‘fix’ them using modern digital tools.
At the same time, my favorite spatial mixes tend to be the ones that expose previously-buried details. Maybe you didn’t notice that interesting guitar part during the bridge in the stereo mix, but now it’s popping out of the left side speaker. So when doing a spatial remix, how do you reconcile the need to maintain some degree of cohesion and familiarity from the stereo master while also taking full advantage of the format to provide the listener a new experience? It seems like these ideas somewhat contradict each other.
Ultimately, what we might call a mistake may not necessarily be defined that way by the artist. But if there’s something that takes me out of the music–let’s say there’s a really noisy guitar part in one of the rear speakers that was masked in the stereo mix–the question I'll ask myself is, “why is it there?” Is what I’m adding to the experience overall worth more than what I’m losing by revealing this particular element? It’s a balancing act for sure.
When I’m mixing in Atmos, I definitely want to create as enveloping an experience as I can. That said, I really do think that every part tells you where it wants to go. If you really listen closely, you’ll always be able to find that place where a certain element is supposed to sit. Once this happens, the listener should be able to just experience the music without any distractions.
So if something's sticking out, some ‘imperfection’ inherent to the source material may not be what’s actually causing the problem. It’s the fact that it’s not blending like it’s supposed to. This is why having the stereo master as a guide can be valuable when the music was originally produced that way. We've been given this roadmap for how the artist and producer initially solved this puzzle, and it can tell us a lot.
"To me, a spatial remix means there’s a defined stereo (or surround) master and then a defined set of source elements I’m to mix from–it could be multitracks, stems, or some combination of both. Typically, I am not being asked to reimagine the song but rather to adapt it into an immersive presentation (Dolby Atmos). What is happening is a spatial re-orchestration of the individual parts to some capacity."
You’ve worked on so many Dolby Atmos mixes, but perhaps the most well-known title in your repertoire is Snoop Dogg’s Doggystyle (1993)–arguably one of the most revered hip-hop albums of all time. How did you come to be involved in this project, and what were some of the challenges involved in doing the immersive mix?
I know a lot of folks who would put Doggystyle as their number-one hip-hop album of all time, so the pressure was on to get it right.
When the project first came to me. John Payne–Senior Vice President of Death Row Records–went back to the vaults and gathered all the tape assets. The record was originally cut to 24-track, edited on tape, and mixed through a console to two-track. By the early 90s, the industry hadn’t quite made the full leap to digital. It was right on the cusp.
So, like many other folks you’ve interviewed, I’m being sent completely raw multi-tracks. There’s no panning, no effects, no processing, no punch edits, and–much to my surprise–no MPC layers. What would happen back then is that the tape machine and MPC would get time-synched together. When you press play, the MPC starts feeding samples directly into the console alongside the tape (kick, snare, hi-hat etc..). These were never stemmed out or printed onto the multitrack.
Anyway, they were able to send me the original 24 tracks for all songs on the record except for some skits and one song that was not able to be retrieved. When I first put up these raw tracks to listen, I was even more surprised. The performances were fantastic, of course, but they sounded nothing like the finished record. Not even close. That speaks to the incredible mixing that Dr. Dre and the team did on this album. So I spent a good four months tweaking the multitrack and constantly comparing with the stereo master because I knew it was my responsibility to make the album sound like it always has.
So the client allowed you to take the necessary time to complete the project as best as possible? I’ve been told that isn’t always the case.
It was Apple that initially approached the team with the project. I told them I could do it, but I needed more time to do it right, and there was no way around that. They granted my request.
When I explained the situation with the missing MPC to some people who understood it, I told them I was committed to going through with the project. Of course, I recognized that just because I chose to do this doesn't mean I get to double or triple my budget, although I'd like to. It became more of a personal mission because I'm either going to do this record 100% or I'm not at all. I would rather not be associated with messing this up, and so I needed time.
My assistant, Andrew Chung, ended up getting pulled into this as well, and it was a massive undertaking. We had to time-stretch things. We had to recreate takes from multiple edits on different reels of tape. And that’s just the restoration work, never mind the remixing piece–recreating every effect and every reverb, every delay, every EQ, and all the compression. This is a heavily processed record.
That takes us back to the issue of the missing MPC, which is no joke. Where is the iconic kick drum in “Gin & Juice,” one of the most important hip-hop tracks of all time? Where’s the snare sample? I tried to see if we could dig up the missing MPC samples, but it was not in the cards at that time.
Editor's Note: The Akai MPC (MIDI Production Center) was an early outboard music workstation that included both sampler and sequencer functions. It was critical to the development of electronic and hip hop music in the late-1980s and early-1990s.
How did you manage to restore those lost samples? Demixing software?
The first thing I tried was stem separation (de-mixing), but that didn’t work at all. It just sounded awful. What I ended up doing–and I asked permission first–is to recreate those sounds from scratch using other samples from my large collection of virtual instruments. Then, using the original stereo master, I created MIDI triggers for each instance so the timing was sample-accurate.
The cool part of doing it this way was that I was already working in Atmos, so I could shape the samples in place. If I wanted the hi-hat to sound good coming out of the wide right surround speaker, I could make the appropriate adjustments to account for the tonal change. It sounded really accurate in headphones–and since this was commissioned by Apple, I knew the spatial audio presentation over headphones was important.
In the end, it all ended up getting approved on their end. But about two weeks out from the scheduled release date, Snoop bought Death Row Records, and therefore his entire catalog. Shortly after, he took everything off streaming. As a result, the mixes sat on my drive, and it ended up coming out about a year later. They made a big announcement about the launch, and it was featured on Apple Music.
The best part of the whole experience for me was about two years ago, when I went to NAMM and Augspurger had a booth. That’s the speaker company that has outfitted all of Dre and Snoop’s studios. They were doing a playback of the Atmos mix on a 9.1.4 array, and a whole bunch of folks–including some of the original team who worked on the record–came to listen.
Everybody had a great time. The speakers sounded amazing. We were supposed to listen to just one song, but they ended up playing the whole record top-to-bottom. Afterwards, a few of those folks came up to me and told me how much they appreciated the level of detail I’d put into the Atmos mix. Listening to the album in California with that group is a feeling I have held onto ever since.
"...I’m being sent completely raw multi-tracks. There’s no panning, no effects, no processing, no punch edits, and–much to my surprise–no MPC. So I spent a good four months tweaking the multitrack and constantly comparing with the stereo master, because I knew it was my responsibility to make the album sound like it always has."
Going back to the original multitracks is high-risk, but also high-reward. You can get so much more out of the experience by having complete control over all the individual components, but–as you’ve outlined–the matching work can be incredibly difficult.
Absolutely. I think that’s a great way of putting it.
What kind of assets do you typically receive from the labels in order to complete these mixes? I know it was multitracks in the case of Snoop, but that seems to be more the exception rather than the norm based on what I’ve been told. It sounds like the vast majority of Atmos mixes are being created from pre-mixed stems.
It’s all over the place. The clients all know that I want as much separation as possible, but it really depends on the stereo mixer’s workflow. If they’re using a heavy bus structure to get the best possible result, then that was the best thing for the music, even if it means I have a bit less to work with in spatial.
Plus, timelines and budgets don’t always allow for mixers to spend five or six hours stemming this out. I don’t like that finances play a factor, but it’s a reality. There have been projects where I’ve personally paid stereo mixers out of my own pocket in order to get more assets. This isn’t something I can do every day, but I'm not going to put something out that I’m not proud of.
But I will say that when it comes to people I’ve worked with, especially over the last several years, the stem deliveries have been beautiful. The mixers are really on it, especially those who work at the label level.
In the case of newer records that were originally mixed in a DAW, I know Pro Tools has a function called ‘save copy in’ that allows you to create an exact duplicate of the entire session folder structure–including audio files, bounces, and the session file. When I was in school taking audio engineering classes, we used to turn in our projects this way. Wouldn’t this be a quicker way of passing assets to the spatial mixer than making stems?
I think the main reason why stems are preferable is because a committed wave file is a relatively future-proof piece of audio, whereas a Pro Tools session is not. Plus, the label may not necessarily agree that the session represents a complete collection of all the assets.
I don’t necessarily need to see someone’s mix laid out in a session file. In my opinion, that’s their intellectual property. But I do love a Pro Tools session with all the tracks clearly laid out and color coded. That saves me a good half hour of prep work. [laughs]. More importantly, it means that I have access to the most assets, which will allow me to be even more detailed in my mixing work.
In an ideal world, I would have access to committed tracks/stems and the original unprocessed files as well. The reason is that there are often decisions made in a production that are specific to stereo, like compression, and having the original tracks as well can allow for immersive specific decisions that really bring the mix to its full potential.
I understand you’re set up to mix not only in Dolby Atmos, but also Sony 360 Reality Audio. How do the formats differ and how does your workflow change?
Sony 360 is its own universe, with completely different deliverables and a unique object-based ecosystem. I’ve never once treated it as a simple format conversion from ADM to MPEG-H or vice versa. Instead, I installed a separate 13-channel Genelec system on the other side of my room.
I've had the pleasure of doing a lot of Sony 360 work over the years, but the amount of content being created in that format is definitely slowing down lately. But it is pretty amazing when mixed properly on 13 speakers–which sadly few consumers get to experience.
That’s been my frustration with the format. They’ve done next-to-nothing to promote 360RA for speakers, it seems like the focus has been entirely on headphones. Nearly every AVR out today supports Atmos, and even a few higher-end models have Auro-3D, but I’ve yet to see one that handles MPEG-H. Is there any way right now for consumers to listen to these mixes on a proper speaker setup?
It is finally possible. At AES last year, they were playing one of my mixes–Olivia Rodrigo’s GUTS album–off Tidal into a new receiver that supported 360RA and had 13 outputs. I couldn’t believe it!
So the 360 mix you did of GUTS is different from the Atmos mix on Apple Music?
Yes, it’s a completely new remix, specifically for 360. The Atmos mix for this album was done by the incredible Prash Mistry. Prash is one of my favorite engineers worldwide, and I was beyond honored that he suggested me as the 360 mixer for this project. He (and his team) also worked closely with me on that 360 mix, giving helpful mix notes to ensure the whole album was consistent across all formats. Dan Nigro (producer), and Matt Morris (A&R) were also heavily involved.
It had to be rebuilt from the ground up because the headphone translation for Sony is different from Atmos. It gives a wonderful sense of space, but there are tonal qualities that certain music really fights against. This is very much a grunge-y pop-punk record, so there’s a lot of distortion that reacts negatively with the headphone filters if not mixed properly. So there was a lot of work involved in getting the binaural experience just right.
Another thing I’ll say about 360 is that the ground speakers are amazing. The format doesn’t have an LFE channel, but it does give you the ability to place bass, kick, and other sub-content below the listener. It’s like being inside a dome of sound.
So you’re not only doing your own immersive mixes, but you’re also mastering other people’s mixes. How does that process typically work? I imagine it must be very different from regular stereo mastering, because the ADM deliverable is sort of a hybrid between a bounced mix and the session format. It’s a standalone WAV file that can be played back through the renderer, yet you can still access the individual object channels after it’s been printed.
Great question. The artist will send me their Atmos mix as an ADM, along with the approved stereo master (if that exists). I’ll then load the ADM into Pro-Tools, so I can see all the beds and objects laid out. Sometimes they've named their tracks, and sometimes they haven't–so I might see it listed out as object 1 through 50, or it could be named tracks like guitars, backing vocals, etc.
My first task is to rebuild their mix and make sure what I’m playing in Pro Tools is 100% identical to what they originally sent. The mapping isn’t too difficult to figure out, but I do have to reproduce their binaural settings. Then, I’ll start rebuilding groups. Is this a stereo stem? Is this a mono file? Is it a 9.1.6 object bed? I can now see that stuff coming from a mile away, and so can my assistant. Organizing things properly is so important because my goal is to get inside their intent.
From there, I’m using combinations of the same process I would use for stereo mastering. It’s not all going through a single bus, but that really doesn’t matter. If needed, I can do linked compression. I’m addressing things like low-end, frequency build-up, spectral adjustments (making a certain element brighter or darker), etc... It really is typically about very gentle equalization gestures or sometimes minor level adjustments.
If I were to rank these processes in order of importance (in mastering), number one would be volume/balance, hands down. The second would be equalization. Third would probably be a tie between compression and saturation–which goes against the common perception that limiting and compression are the most important facets of mastering. Ultimately, these are equally important in their own way, and just like stereo mastering, it is about making subtle moves that serve the music, and ensure translation.
With Atmos and other object-based formats, it’s important to check your mix on both headphones and speakers. It seems like every mixer working in this space has their own process: some start on speakers and finalize on headphones, while others work predominantly in headphones and finish on speakers. Some have even told me they ignore the binaural aspect altogether and focus only on the speaker mix. How do you go about this, and do you ever find yourself making concessions in the speaker mix in order to maximize headphone compatibility?
I pretty much always work with speakers first. The speaker render is the most honest representation of the immersive intent, and so that is where I want to make my initial decisions. It is also the most inspiring and creative environment for me. I also find it is much easier to fix a problem accurately in the immersive speaker environment versus on headphones. I do, however believe that headphone translation is extremely important, and so I spend time on headphones in all of my mixes and masters, ensuring translation.
I don’t think anyone doing this type of work has any expectation or delusion of grandeur that the headphone rendering will perfectly reproduce the spatial mix as heard on a proper speaker array. At the same time, we know that most of the audience for these mixes will be listening on headphones. I’m never going to deliver something that I wouldn’t be proud of, so it needs to sound amazing in binaural/spatial as well.
The more work you do, the more you’ll realize there are definitely ways to mix on speakers that will help with your binaural translation. I think hugging the walls is usually a good idea because having de-correlated content helps with a successful fold-down.
I’m mixing on an 11.1.6 system with full-range Lipinski monitors all around. It’s a gorgeous environment. Those speakers are ruthlessly clear and accurate. They make me work hard! Once I’ve got a mix going on the speakers to the point where I’m happy, then I’ll put on my Audeze CRBN electrostatic headphones.
They're a little bit light in the high-end, but super-revealing and really put me inside the music. So I’ll make some subtle tonal adjustments on those or just take note of anything poking out too much. I find that usually the headphones are accurate in this regard. If the vocal sounds slightly too hot, I’ll go back and check the speakers–and surely enough, it is actually a dB or so too loud.
Then I’ll throw on my LCD-5s, which work for sibilance and top-end EQ. I’m rarely making adjustments greater than 0.5 or 1 dB. If I feel like I need to do more than that, I’ll take the headphones off and go back to the speakers. 99% of the time, the best way to solve the problem is at the source, which for me is the speaker render.
Are the headphone codecs perfect? Absolutely not. I do not, however, think that that is an excuse for not doing my very best to ensure my mixes sound good on both speakers and headphones. Some mixes translate more seamlessly than others, but at the end of the day, I am never going to send something that I don’t believe sounds good, at least to me, in both.
I assume you also have a pair of AirPods Pro or Airpods Max to check the Apple binaural?
Yep. I’ll load the MP4s onto Dropbox, then listen to those as I’m going for a walk. Getting out of the chair helps give me some perspective. If something’s poking out, again, I’ll go right back to the speakers. If the backing vocals sound too loud in the headphones, nine times out of ten, they actually are just a little too loud.
I am also now using the binaural renderer by Audiomovers, or the Immerse Virtual Studio by Embody, so I can check things in real time.
In your various trials and tribulations of doing this work, what do you find are common examples of spatial mixing decisions that don’t translate well to headphones?
I think that we need to be very careful about pulling things off the wall. If you bring an object further out into the room, you’re asking multiple speakers to reproduce the same sound. It can be pretty forgiving in space, but on headphones, you have a bunch of very complex filters that are going to treat that sound differently. This can sometimes lead to problems like a build-up in volume or comb-filtering.
I also think that people expecting height information to be exceptionally apparent on headphones is asking a lot of binaural at this stage. We’re just not quite there yet. That does not mean we should not mix height information (I love mixing content up high), but we should be aware of the limitations in current translation when making arrangement decisions.
Even disregarding the headphones for a moment, I’ve never liked the idea of panning things out into the room in a surround mix because the listener’s perception of your decisions becomes dependent on how accurate their setup is. Not every listener is fortunate enough to build a dedicated music listening room–it could be a multi-purpose space like a family room or an office with its own set of limitations, like having a door or window where one of your side speakers is supposed to go. Maybe the couch is against the wall, so there’s no place to put rear surrounds. Perhaps the height speakers are wall-mounted rather than recessed in the ceiling.
By sticking sounds directly to the outer edges of the space–like a guitar only in the right rear channel or a vocal in the center–you’re ensuring that those decisions translate even to the least ideal speaker setup. If you were to pan an object halfway out into the room and then a quarter of the way up towards the heights, I doubt anyone would actually be able to perceive that.
When it comes to pulling something off the wall, the question to me is why. What is this supposed to achieve? From a speaker standpoint, it gives the element a sense of size. We’re also bringing the source closer to us, which can also be achieved in other ways. One could simply make that element louder or adjust its tonal presence by adding or reducing reverb.
So I very rarely pull things off the wall, but that’s not to say I don’t want to give the listener the experience of it being pulled out into the room. I’ll achieve that using various approaches to decorrelating sources instead because that usually translates better in all cases.
From your vast catalog of immersive work, are there any specific tiles you’d recommend to listeners? I know Snoop Dogg’s Doggystyle would be one.
For sure. I’ll give you a few others, though I do have a playlist linked on my website.
There’s a brand-new record that just came out by a guitarist called Marcin. He’s a brilliant, virtuosic guitar player. I just finished this one, and it’s epic–there were hundreds of guitar tracks per song. It’s called Dragon In Harmony. Next, I would put up a record from last year called Grief Chapter by Mother Mother.
Are these major label or indie projects?
Those two are both label, but I’ll give you some indie ones as well. Eliana Cuevas’s Sere Libre–this was a gorgeous orchestral project. We actually did a separate cinematic Atmos mix that was released in movie theaters. The visuals that went with it were beautiful.
There’s a record by Go Go Morrow called Ready that came out great as well. Finally, one that I always go back to is Jonathan Kawchuck’s Everywhen. That was an early project specifically made for 7.1.4, and it’s a wild ride.
When it comes to other people’s Atmos mixes, do you have any favorites?
This is a great question. To be completely honest, outside of work I listen mostly to Blu-Rays and classical music. So Nidarsodomens Jentekor & Tondheim Solistene’s Lux (recorded and mixed by Morten Lindberg) is definitely a favorite. I also adore the Soundtrack of the American Soldier, which was recorded and mixed by Leslie Ann Jones, and mastered by Michael Romanowski. It is not in Atmos, but if you ever get to hear the God of War Ragnarok soundtrack in Sony 360 (on 13 speakers), it is out of this world. That was mixed by Eric Schilling, and produced by Herbert Waltl.
For non-classical music, I really like Steven Wilson’s latest record The Harmony Codex and the new Peter Gabriel record (i/o) too, which was mixed in Atmos by Hans-Martin Buff. Live in Munich by the Bob Reynolds Group is a new release mixed by Kseniya Kawko that I really like. Human Being Kind by Dave McKendry, mixed by Stefan Bock, is absolutely beautiful.
One more I will add (I could go on!) is the The Diary of Alicia Keys. If you hear that album on a proper immersive system, it is stunning. Anne Mincieli, George Massenburg, Eric Schilling and Michael Romanowski outdid themselves on that record.
If you could mix any record–new or old–in Atmos, which would you choose?
My first choice used to be D’Angelo’s Voodoo, but it was recently mixed in Atmos (and done well!). So I think my new answer is John Mayer’s Continuum. I think it is an engineering masterpiece, and I have a feeling it would sound stunning in immersive. Plus, I know that music really well. I’ve been playing those songs in bands for years.
Next would be Radiohead’s Kid A. If that happened, I would likely lock myself in a studio for 6 months, as it would be another album which would deserve the greatest care imaginable. Another one I would love to do is Bruno Mars’ 24K Magic. There are so many layers in that record, and it hits really hard and already feels three-dimensional, even in stereo. Lastly (there are so many I would love to do), I would love to mix the Bela Fleck & The Flecktones Live at the Quick album in Atmos.
The catalog of Atmos music on streaming services right now is just staggering. It’s so far beyond what was previously available on quadraphonic vinyl/tape or 5.1 disc
Yeah, it's a beautiful thing. I know some people say the public at large doesn’t care because they’re listening to music on iPhone speakers or earbuds, but I think there’s plenty of listeners who deeply care about high-quality sound and these immersive formats. Listeners today are very fortunate to have access to this.
What do you think about more-economical speaker-based systems, like the Sonos Arc and ERA-300?
The more discrete placements of a speaker you have, the closer you’re getting to the original intent of an immersive mix. But I would never discourage anyone from listening with a soundbar because we have to remember philosophically that we are creating object-based music.
If you go the route of creating channel-based music in formats like Auro-3D or just straight PCM, then the listener has no choice but to build the appropriate system in order to hear your mix properly. If we choose to use Atmos, we are automatically signing the waiver at the door that this is an object-based adaptive format and it will play back to any device that is Atmos-capable. To me, that is the true power of the format and something we need to understand as mixers.
I prefer not to measure the experience these devices offer in terms of a discrete experience but rather envelopment. If you don’t actually have rear speakers, then the perception of having things come from the rear probably won’t be great. But did the listener think, “Wow, this song is bigger than I’m used to hearing?” Did it feel like the sound extended beyond the confines of the devices? As long as it’s moving people in some way, I’m happy.
I completely agree. I have a neighbor with a Sonos Arc and two rear speakers, and the experience–while not equivalent to 7.1.4 by any means–was surprisingly enjoyable. At the same time, when you listen to how intricate some of these mixes are–how much the panning decisions have been labored over, like having specific sounds come from the side, rear, or height speakers–it is a little disappointing that so few people get to experience that.
That’s just the nature of the time we’re living in. But I’m a hopeless optimist, so I’m always focusing on the glass-half-full side. Things continue to improve year-over-year, and I feel very fortunate to be doing this kind of work.
