double-blind testing is a cornerstone of the scientific method, and i have difficulty understanding why it is considered so heinous among audiophiles. if it is difficult to carry out in practice, and many such efforts are flawed, that does not invalidate the technique or its potential usefulness. therefore, my present position is that double-blind testing must be dethroned by a superior method of evaluation, and as i see it, non double-blind testing seems riddled with intrinsic flaws that are inherently uncorrectable, as long as human beings are involved in the evaluation process.
even more difficult for me to understand is why the flaws of current non-double-blind testing are not so readily apparent to audiophiles. i would like this topic to be free from vitriol, and i see a lot of ego and belief-system defenses render this topic so emotional as to be unproductive. therefore, i will assume that every reviewer on this site is beyond reproach, ethically, and i the concerns i have are not directed at them, but rather are directed to the flaws inherent in non-double-blind testing.
first off, among non-holy men, money is a powerful corruptor; this need not be elaborated on. for a review to be meaningful, it must be free from the potential influence of factors such as money, reviewer-manufacturer friendship, advertising considerations, whether the reviewer got a sweet deal on his equipment, etc. let me give you an example of this that i face. i am a physician, and it so happens that i carry medications in my office for my patients when 1) i believe them to be superior to products available elsewhere to the patient and 2) when i find that generic products are being sold to my cash-paying patients at a charge many times what i pay for the product myself. my policy is to simply give the products away when they are low-priced (usu example #2)- say under $10-15, when the patient will be faced with paying $50 or more- or i show the patient an invoice of what i paid for the product and give it to them at my price. the reason? i cannot afford to have the patient question my motives for recommending the product i carry-even the appearance of a possible financial influence is unacceptable to me. i don't want the patient asking himself, "did he advise this for my benefit or because he made a buck?" the patient paid me to do my absolute best for him, just as i paid the subscription cost to have the reviewer do his best for me, and in neither case should there be any question of additional, non-transparent motives or forces at work.
similarly, i do not appreciate that when i read a review, i have to wonder whether the reviewer got a good deal on the sample he tested, whether he was of the belief that tubes are inherently better than transistors or v/v, and this subtly influenced his opinion, whether his buddy is the manufacturer, or whatever. of course, this must occasionally happen, even if it never has and never will on this site. the possibility of bias and influence that i am unaware of, and which the reviewer may even be unaware of, casts doubt on the reliability of the process itself.
i'm sorry, i am old enough that for me to believe that i should place blind faith that the reviewers will not fall victim to the many potential pitfalls of a reviewing system that is inherently flawed is not something that i am inclined to do. and seriously, can it be argued that with the potential for such bias, the system is not inherently flawed? to believe in this process requires that i have faith that the reviewers are not just incapable of corruption, but also immune to a myriad of psychological influences that affect pretty much every human being, to some degree. how many priests, doctors, lawyers, politicians, executives, etc have proven, to our surprise, to be corruptible, that we should expect that ANY group of people are incorruptible? that is utter naivete.
i will await some attempts to see results from double-blind testing, and i admittedly am no authority on its flaws as applied to audio. the forums i have read usually just degenerate into name-calling and insult-slinging between the two camps, and there is not much meat, just a lot of hot sauce to that recipe. do the reviewers even acknowledge the above potential problems? all i see is offense taken that their integrity is being assailed. which, in fact, it often is-like i said, the debate is unpleasant and largely unproductive, as far as i have seen. i have no qualms with the reviewers, just the system that, as i see it, is intrinsically flawed, and as a reader it leaves me unable to place much faith in the reviews, just as i would expect one of my patients would question whether the $40 medicine he purchased from me was influenced by factors other than what was best for him, as a patient.