In Reply to: Huh! But there are so many things to "prove". Look at Norm and Quint's explorations... posted by clarkjohnsen on December 27, 2005 at 13:59:19:
I think it's obviously ridiculous to expect a customer to do DBTs before they can make a claim that they think something makes an improvement or not. If anyone is going to do a test it should be the manufacturer but even that has problems in a lot of cases. it would be ridiculously prohibitive for small manufacturers.The sort of testing that a manufacturer is likely to do isn't going to be DBTs in most cases. It's going to be performance testing and I think some performance testing should be expected to back up performance claims. Even with that, however, I think there are always going to be products where the user simply has to 'suck it and see' but we all do that to some degree even with components where we may have reliable electrical test data as to how the component performs. No test results of any sort are ever going to substitute completely for one's own impressions.
Life does get messy when it comes to some tweaks, however. A manufacturer can make claims of 2 sorts: claims about how the product affects what you hear, and claims about how it works. Listening tests could support the first sort of claim but not the second, and the simple fact is that whether or not the second sort of claim is true need have no bearing on the truth of the first sort of claim. The things that work will work regardless of whether the explanation given for how they work is true or not. Unfortunately more than a few claims for how something works simply seem ridiculous—they refer to theories that aren't accepted by science, or problems that no one else has identified with our systems and playback, or both. Such products are definitely magnets for extreme scepticism regarding their claims on how what you hear will be affected. After all, if one set of claims is unsupportable with accepted science, then there is always going to be a tendency to strongly doubt the other set of claims as well.
I think manufacturers need to be cautious about what they claim. I don't have a problem with a manufacturer saying something like "I discovered it by accident, I don't know how or why it works. but the effect is consistent and repeatable and what you hear is …". The customer gets enough information to know what to listen for in order to satisfy themselves about the claim and no-one is going to get involved in arguing too much about the claims made for how it works. Any discussion about that is clearly going to be speculation between the parties to the discussion and the manufacturer isn't making claims which, if proved wrong, may adversely influence judgements regarding the performance claims. The performance claims are simply left to stand or fall on their own.
And as far as I'm concerned, all the customer has to say is whether or not they believe they hear a difference, whether that difference occurs reliably or not with use of the product, and whether or not they think it represents value for money. After all, that's the sort of information you get from a friend when you ask them for comments on the car they drive or some other product they use, and no-one normally asks friends or other product users to provide test data to support their view. We assess their views on the basis of what we know about the person expressing them, their knowledge or experience and any other relevant qualifications they may have which would give extra weight to their opinion, and whether or not we trust them. I don't think user claims in relation to audio need to be assessed differently than that.
David Aiken
This post is made possible by the generous support of people like you and our sponsors:
Follow Ups
- Re: Huh! But there are so many things to "prove". Look at Norm and Quint's explorations... - David Aiken 21:14:02 12/27/05 (4)
- "If anyone is going to do a test it should be the manufacturer." Hmm... - clarkjohnsen 08:06:57 12/28/05 (1)
- Clark, I'll disagree with some of your points… - David Aiken 14:25:07 12/28/05 (0)
- Well, we seem to agree again... - Silver Eared John 23:21:14 12/27/05 (1)
- Hair splittin'... - David Aiken 13:31:09 12/28/05 (0)