TUTORIAL - Audio enthusiasts vs. the Double Blind Test

How can we expose more people to critical thinking?

Was this useful?

Yes
10
83%
No
0
No votes
STFU JJ
2
17%
 
Total votes: 12

En folkefiende
Posts: 17511
Joined: Thu Aug 12, 2004 4:38 am
Location: Waiting for an electrician

TUTORIAL - Audio enthusiasts vs. the Double Blind Test

Post by En folkefiende »

In the audio industry, there is a vocal subset of enthusiasts, artists, recording engineers, and other engineers who claim that "Double Blind Tests hide small acoustic differences".

This is important, at least to the audio industry, for several reasons.

The first is that the auditory system is probabilistic. As in any probabilistic detection, you can tune it from "almost never miss a difference at the cost of detecting a lot of nonexistant differences" to "almost never detect a false difference at the cost of missing a lot of actual differences", or anywhere in the middle. It seems quite clear, having run some way too many audio tests, as well as by reading the literature, that the auditory system is tuned near to the first, "always find differences, even if they aren't really differences". If you want to argue this, let's do it in a different thread, but there is rather a great heap of evidence, pubished and unpublished. What does this mean? It means that in any given situation, listening to two things, you're most likely to hear a difference, due to simple probability.

The second is that the auditory memory is built in approximately three steps, like most human memory. The lowest (first) is "acoustic memory" or "loudness memory", a short-lived sensation that dissolves within 200 milliseconds or so, that compares the actual partial loudnesses out of the cochlea. (Note, loudness is the sensation, i.e. internal representation of a sound. Intensity is the external, i.e. measurable waveform, representation.) That is condensed, a very lossy process (1000:1 or so) into auditory objects or concepts, something that sticks around for a few seconds or so, the quality of a 'click', etc. That process is then reduced again to long-term memory, losing a lot of information in the process, again (1000:1 or so again), resulting in something like a few bits/second or so of long-term memory, if the listener chooses. This is a problem for two reasons, first, because the process is hugely lossy, and second, because the reduction of loudness memory to auditory object can be guided by the mind. This guidance, by simply doing different things on two listenings of the same thing, and by having different focus, can mean that you actually did hear different things, and will remember different things.

The result is that in the absense of a way to falsify what the listener reports, there is very little confidence in any particular report. Quite obviously, a blind test or a cognate thereof, is of great use. As in many other kinds of testing, experimenter expectation can affect the listener if the experimenter is at all in the sensory range of the listener during the test, so double-blind testing is, as perhaps expected, called for here.

Some people, it appears, do not like to accept this knowledge (I hesitate to call anything "fact" but the evidence here is rather overwhelming), and routinely make statements about sensation that the person versed in auditory testing find very questionable.

Herein lies the problem: The artists and material producers for the most part come from the anti-DBT crowd, and the science around the subject from the second part. Ergo, there is a great deal of upset and confusion, a lot of smoke and fire, but very little clear light shed on most subjects.

Into this void comes the high-end equipment manufacturer, a subject of another thread, perhaps.
Formerly jj, the enemy of the people, aka the bullies who rant and lie here.

User avatar
xouper
Posts: 9473
Joined: Fri Jun 11, 2004 4:52 am
Location: has left the building

Re: TUTORIAL - Audio enthusiasts vs. the Double Blind Test

Post by xouper »

jj wrote:In the audio industry, there is a vocal subset of enthusiasts, artists, recording engineers, and other engineers who claim that "Double Blind Tests hide small acoustic differences".
How do they know this is true?

Do they have any double blind tests that back up their claim?

But seriously, can you tell us more about their justification for making that claim? If we are to discuss the flaws in their argument, it would help to know what they are arguing. Or is that not the point you were making?

En folkefiende
Posts: 17511
Joined: Thu Aug 12, 2004 4:38 am
Location: Waiting for an electrician

Re: TUTORIAL - Audio enthusiasts vs. the Double Blind Test

Post by En folkefiende »

xouper wrote:
jj wrote:In the audio industry, there is a vocal subset of enthusiasts, artists, recording engineers, and other engineers who claim that "Double Blind Tests hide small acoustic differences".
How do they know this is true?

Do they have any double blind tests that back up their claim?

But seriously, can you tell us more about their justification for making that claim? If we are to discuss the flaws in their argument, it would help to know what they are arguing. Or is that not the point you were making?
Unfortunately, the justification varies by person, from "everybody knows" to "professionals don't need to do that" to "we know it hides things because I heard them hidden when I tried the ABX test", and so on.

For many such people, it appears to be an element of faith. For some people, I believe it's access to a market segment.

DBT testing has, of course, gotten down to the level wherein subjects detect signals very commensurate with the levels predicted by physics and mathematics. That leaves very little room for error, in my book.
Formerly jj, the enemy of the people, aka the bullies who rant and lie here.

Earthborn
Posts: 75
Joined: Mon Jun 07, 2004 4:02 pm
Location: Terra Firma

Post by Earthborn »

I think it is possible to test this with a 'triple blind' experiment: perform a double blind test, but don't tell them that it is a double blind test. ;)

You show them a player and a switch and someone who turns the switch. The person turning the switch doesn't know that the switch is fake. The actual audio comes from a second set with a different switch. If the test subject can't hear the difference between the two audio signals, you know that either there is no difference, or the double blind procedure makes it imperceptable. If the person does hear a difference, but always claims to hear the better sound when the switch he sees is turned to the signal he thinks should sound better, then you know the knowledge of which signal he's listening to influences perception. If the person does hear a difference, but claims to hear the best significantly often when he hears the signal that is supposed to be the best, then you know there really must be a difference.
"Mulder, the Truth is Out There, but so are Lies!"

User avatar
MRC_Hans
Posts: 519
Joined: Fri Jun 04, 2004 8:11 pm
Location: Denmark

Post by MRC_Hans »

I cannot vote in the poll, because I do not understand what the options stand for; what exactly am I saying yes or no to, and what does the third option mean? Is it a special Planet X code? (otherwise the Planet X option is missing ,and I refuse to vote anyway)

Seriously, though, it is not difficult to understand why audiophiles claim that tests are not adequate. If they were to admit they were, they would be unable to justify the exorbitant prices on the stuff they sell or promote; entire industries would go broke, and numerous magazines would fold. It is the livelihood of an entire community to perpetuate the illusion that anybody can hear the difference between a 500$ and a 5,000$ amplifier without first looking at the price-tag.

Edited to add: And of course those who bought the expensive equipment will swear there is a difference. Admitting otherwise would be the same as saying loud and clear: "I AM A SUCKER!".

Hans
[i]Fly pretty, anyone can fly safe...[/i]

En folkefiende
Posts: 17511
Joined: Thu Aug 12, 2004 4:38 am
Location: Waiting for an electrician

Post by En folkefiende »

MRC_Hans wrote:I cannot vote in the poll, because I do not understand what the options stand for; what exactly am I saying yes or no to, and what does the third option mean? Is it a special Planet X code? (otherwise the Planet X option is missing ,and I refuse to vote anyway)
Hans
The question was stated as: Was this useful?

Obvious, this refers to the article below it. The third option is the same as "go away, stupid". It means Shut the Up.
Formerly jj, the enemy of the people, aka the bullies who rant and lie here.

En folkefiende
Posts: 17511
Joined: Thu Aug 12, 2004 4:38 am
Location: Waiting for an electrician

Post by En folkefiende »

Earthborn wrote: The actual audio comes from a second set with a different switch. If the test subject can't hear the difference between the two audio signals, you know that either there is no difference, or the double blind procedure makes it imperceptable. If the person does hear a difference, but always claims to hear the better sound when the switch he sees is turned to the signal he thinks should sound better, then you know the knowledge of which signal he's listening to influences perception. If the person does hear a difference, but claims to hear the best significantly often when he hears the signal that is supposed to be the best, then you know there really must be a difference.
In my experience, the visible switch will "rule". I have, in fact, run a deliberately non-blind test. Subjects, even thouse who are trying not to be influenced by the visible switch, will be influenced. Even someone who is told that the switch is bogus seems unable to ignore the influence. The effect of sighted testing is indeed that strong.

As for the "real" test in your proposal, you'll find that not giving the subject a known "reference" at all times will desensitize the testing by some small but measurable amount. A subject does best if they have at least one constant "reference" signal throughout. It doesn't even have to be "the original", as long as it's pretty good.
Formerly jj, the enemy of the people, aka the bullies who rant and lie here.