About This Null Test Business
Whenever it’s about the subtlest and most controversial issues in audio, the so-called null test or difference test isn’t far away. Therefor it surely deserves an article on The Science of Sound.
There is a famous German proverb among engineers: “Wer misst, misst Mist.” Other than being a nice play with words, what it says is that as long as you try to measure anything, there’ll be a whole bunch of crap obfuscating your measurement. In other words, chances are good that your results don’t give you the information you wanted to acquire in the first place, but are dominated by something totally different.
As a consequence, engineers spend much more time with interpreting and confirming results than they do actually with measuring. The key is always in interpreting the data.
A good example to illustrate this is the famous null test or difference test. It’s a simple testing method that can help prove that two signals are more or less exactly equal.
Null Test Philosophy
Audio enthusiasts are sometimes very obsessed about processes that are supposed to not alter audio in any way. Especially in the notoriously prejudice-laden realm of digital signal processing. At some point usually, a null test will enter the discussion. Either to prove that there is nothing to worry, or to prove that the world is ending and it’s impossible to make good music today because digital.
Over the years I’ve grown very skeptic on the inflationary use of the null test, as it doesn’t have much consequence when it comes to actual sound. The only useful reason for it is to prove that two signals are exactly identical. And honestly, we shouldn’t obsess so much over identical or near-identical signals when there are so many exciting ways to make signals different. But of course, we live in a time where it’s en vogue to hear a difference where there shouldn’t be one.
How to do a Null Test
Anyway, there are some cases where this test comes in handy. And luckily it’s quick and easy to do.
All you have to do to test if two signals are identical is to mix them together while flipping the polarity of one of them. If the result is pure silence, the two signals are equal.
Well, silence is relative. If you don’t hear anything, that doesn’t mean that there really is nothing left at all. Maybe you just need to turn up the volume.
There is only one case where the test is really meaningful, and that’s when – in a digital system – you get complete digital silence. In other words, an audio file full of zeros. That means that the two audio streams were really identical bit by bit. As a funny side note, I was actually once confronted with claims that two bit-by-bit identical audio files can still sound different because quantum mechanics or something. But I digress.
The thing is: in most of the really interesting cases, the null test fails!
These interesting cases are the ones where it’s technically impossible to achieve perfect equality, but a best possible match – soundwise – is still desired. Good examples are A/D and D/A converters or emulations of analog gear. Or simplified, anything that involves analog signals in any way.
In these cases, the null test virtually always fails miserably. But that doesn’t necessarily mean there’s something to worry about.
The problem is that there are countless reasons for a failing null test that don’t affect the actual sound in any way. That’s boring stuff like slight level changes, time delays or slightly non-flat frequency responses.
That takes us back to the beginning. When doing a null test under non-trivial circumstances, you need to take extreme care to get this stuff right so it doesn’t obfuscate your results.
How to Interpret Null Test Results
So let’s walk through some of the possible ways in which a null test can fail. Assume you made this test and you still get a difference signal. What can we learn by inspecting the difference?
First of all, the level matters, of course. If it’s so low that you can barely hear it even if you crank up your monitor volume to dangerous levels, it’s likely that the difference is irrelevant. However, the only way to really prove with some confidence if there is or is not an audible difference is a proper blind listening test. Including all it’s time- and nerve-consuming glory if done right.
However, if there is some difference left, there is still some information to gain from it about what kind of phenomena could be responsible for it.
The first question is: Does the difference signal contain music?
If the music signal that you were originally using completely disappeared and the result contains only noise, congratulations! You’ve got a nearly perfect process in front of you that just adds some noise. However, that doesn’t happen very often.
In most cases, there will be some recognizable portions of the original signal left. But how does it sound?
If the overall spectrum is similar to that of the original signal, it’s likely due to a slight level difference. If you play around with the signal levels, you might be able to bring the difference down a bit more.
For a little bit of context: a practically inaudible level difference of only 0.2 dB will yield a difference signal of around 32 dB below the original signal. This will be audible even at moderate listening levels.
If the difference signal sounds hissy with much content in the high frequency range, this is usually due to a slight time shift between the two signals. Try to align the two signals better using small delays in your DAW.
Only one sample of delay can lead to a difference signal that’s only a few dB below the original in RMS level. If the time difference is even larger, you get awfully sounding comb filter effects on top.
If the spectrum of the signal sounds strongly colored, there’s probably a slight non-flat frequency response at play. This can be a bit misleading, because even slight changes in frequency response are greatly exaggerated in the difference signal.
So even if the difference signal sounds strongly colored, the real difference can be absolutely inaudible. For example, an EQ setting with only +-0.5 dB gains is hardly audible at all. A null test with this treatment will however lead to a difference signal that’s colored with a frequency response in a range of more than +-10 dB.
The same goes for the effects of harmonic distortion. Since the difference test removes the undistorted signal almost completely, there are only harmonic distortion products left to hear. And that really sounds awful. Even if the distortion along with the original signal is inaudible.
Another similar example is dynamic compression-like processing. Within a null test, transients and onsets will sound exaggerated in the presence of even slightest dynamic gain changes.
Note also that any saturation distortion, both soft or hard, goes along with some amount of compression. So as a result you will getexaggerated harmonic distortion products combined with overly snappy transients.
At this point, it’s probably best if you go ahead and try some things for yourself. So fire up your DAW and load a song or instrument recording onto a track. Send it to the master bus directly, as well as a second copy through a bus channel. Invert the phase in the bus channel and you should get total silence.
Now insert some EQs, delays, compressors or saturation plugins into the bus channel and dial in the subtlest possible settings. Listen to the difference signal (turn up monitor volume if necessary, but be careful when doing so!) and hear what happens.
And of course, don’t forget to share your results in the comments!