Home Propeller Head Plaza

Technical and scientific discussion of amps, cables and other topics.

[ Follow Ups ] Thread: [ Display  All  Email ] [ Propeller Head Plaza ]
[ Alert Moderator ]

Done Homework

You said: "A whole bunch of people (me, jj, Geddes, Kal R, to name a few) have cited papers here."

I've done searches for posts in all forums containing either "DBT" or "double blind" in the text from you, JJ, Earl Geddes, and Kal Rubinson. No date limits were specified in the search criteria.

Results:

Posts by you: you occasionally throw out a mention of an author like Fletcher, Floyd Toole, Ted Grusec, Gunther Theile, Ernst Schroeder, and "all those guys", and you occasionally link a phrase describing the topic of particular research with a specific author. None of the posts I looked at as a result of the search contained a specific citation or any more specific reference than an author's name and, at most, a topic. I have to admit that I didn't check every post that came up because I got tired of looking at 2 line jibes at others which contained no specific info so, after a while, I started choosing which ones I looked at based on whether the opening line or so quoted on the search page was an attack on someone or whether it looked like you might actually be saying something.

Posts by Earl Geddes: few posts met the criteria and none contained a specific reference.

Posts by Kal Rubinson: few posts met the criteria and none contained a specific reference. Among Kal's posts was a comment that he considered DBT essential for a statistically reliable determination but other statements included one to the effect that some things were readily audible under DBT conditions and some not, and another referring to the use of DBTs in some instances stating that anything that can be done, ie the tests, "can be done badly and usually is". In many ways I suspect I may not be too far from his position since my view remains that they have their uses and they have their limitations, they may not be a good test for showing up some things, and the simple fact that a test is DBT doesn't necessarily make it a good test. It still has to meet the other requirements for a well conducted test and it's unreliable if it doesn't meet those other requirements.

I did come across an interesting thread involving you and Jim Austin in which Jim seemed to be taking a similar stance and also pointing to mathematical problems and stating his view that they were an unreasonably strict test. Obviously I had some sympathy with his views too.

I then searched on JJ using the same search parameters—as expected these 2 searches produced the most results and I ended up picking and choosing based on the text quoted after each item in the search list. The results were much the same as for your posts: references to authors but no specific citations or recommendations that I came across.

So, my conclusions at the end of all of that is that none of the people you mentioned, including yourself, have made a citation of a specific reference in any post including the phrases "DBT" or "double blind". What has been made are references to specific researchers and occasionally references to specific topics in their research.

Note that all such references where a topic is mentioned seem to refer to tests conducted under DBT conditions rather than references specifically providing support for the use of DBT in audio type testing of the sort discussed here.

If we're going to discuss what the limits of audibility are or whether or not some characteristic like a specified distortion or a particular variation from a flat frequency response makes an audible difference, I have no qualms about the use of DBT. When it comes to comparisons of specific products I start to get reservations which tend to be supported by the test results I see. In part that is no doubt due to the fact that we all tend to resist findings we don't like, but in many cases my major reservation comes from the fact that the tests, as reported, either don't seem to have been designed or done well or there is simply not the information provided to enable one to see whether they have been designed and/or done well. I came across JJ frequently saying in the posts I looked at, and I used to admire him for actually saying this when he was around and I was reading his posts and debating with him on occasion, that DBTs like any other test have to be done well if they are to have any value and that it isn't easy to do them well. Kal made that point too plus he indicated that he believed that some things weren't readily audible under DBT conditions and the possibility that might be the case has been one of my concerns. I wonder whether Jim Austin was saying the same thing as Kal when he observed that he thought DBTs might constitute an excessively demanding standard.

JJ also frequently made the point that DBTs weren't necessary for determining preferences and preferences are heavily involved in a lot of the debates I've seen here where DBTs have been invoked by some. JJ seemed particularly proficient in separating out the preference issues from the other issues in his posts and dealing with each separately, and while I had reservations about his position on some things I genuinely appreciated his contribution on a lot of things and I miss his presence here in AA. I say that even though I clashed with him very strongly on one or two occasions.

I suspect I might well feel a lot more positive about DBTs involving audio products/tweaks and the kinds of things often debated here if I was satisfied that the tests being reported were being done well enough to satisfy the kinds of standards routinely expected of research in the peer reviewed journals. Note that I'm not asking for the tests to actually be published in such journals, but I am asking for the information provided about the tests to include the information that would be required if they were to be published in such a journal. Reports of test results which don't indicate the number of subjects, don't specify the music used for testing or why it was chosen, which don't describe the test procedures and conditions used, and which don't report enough of the data to enable the reader to calculate the results for themselves if they choose to do so simply don't cut it for me. Effectively what they contain is no stronger than the anecdotal evidence you refuse to accept. The missing data doesn't have to be provided on the page where the test is reported—many people simply wouldn't read the page at all if it was—but there should at least be a link to another page or a reference to a publication containing the missing information. And I'd also feel more positive about DBTs, or perhaps just some DBT supporters, if they happened to be as fastidious and honest as JJ was about spelling out just what failure to estabish an audible difference with a DBT actually did mean, and it isnt't that there is no audible difference. There seems to me to be a world of difference between the claims JJ made for DBTs and the claims made by many others here.

To put that in psychological terms, the bad impression I get from reading reports of DBT tests which fail to provide essential information, and from reading many of the pro-DBT posts here which definitely fail to approach the standard JJ set for intellectual and professional honesty are probably exerting a halo effect when it comes to my feelings about DBTs generally.

So that's if for homework for me at the moment. You seem to be wrong about citations of references but that could be because you and I are using the word differently. I meant the sort of references that $orabji provided, references to specific papers, and you may have meant references to specific authors/researchers.

I'll start chasing some of the papers/books $orabji mentioned in a few days.

David Aiken


This post is made possible by the generous support of people like you and our sponsors:
  The Cable Cooker  

Follow Ups Full Thread
Follow Ups
You can not post to an archived thread.

[ Top Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2025, Audio Asylum, all rights reserved ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]