more 'snake oil'
You have answered your own question...
I should say I don't agree with your question or your answer.
Anyhow, WHF? staff have already told us many times - even in this thread - that their reviews are subjective so it's pointless devising a test to prove it!
Well Chebby if you have read my post and come up with that response then I have clearly over-estimated your IQ. 
So it's just a piece of "puff" then! The problem is that there are doubtless some of your readers who believe what they read and possibly spend a lot of their hard-earned money on that basis. Might I suggest that you owe your readers a bit more than that!
My cynical side says that there is no incentive for the industry (and that includes the magazines) to find definitive answers because they would have less to sell / write about. Maybe I'm being too harsh but ...
Chris
PS I'd be happy to take part in a listening test but only if it were scientifically conducted, ie double blind and statistically significant.
If those are your conditions, Chris, I fear we won't be seeing you down here. That is not the point of these days, as I explained. (Although you're still very welcome to come and see us for a BQ - I'm sure you'd find it interesting to see how we work.)
You'll be aware, no doubt, that WHF has never done testing like that. As has been mentioned or at least hinted at a couple of times on this thread, what we try to do is give an informed and experienced opinion on the equipment we review, in as real-world a way as possible. And through that help people come up with a shortlist of likely products that will suit them.
Our team has many, many years of cumulative experience behind them, and we are as consistent as we can be in testing all the kit that comes through our doors: everything is reviewed in-house by our full-time team; everything is looked at by at least two, and often more, members of the team; all reviews are then checked to ensure that what's written up matches what was decided at the reviewing stage. That way we have at least some system of checks and balances to allow for the frailties of human nature.
But no, it's not bench tested at any stage.
So yes, ours is a subjective rather than an objective opinion. But it's one with decades of experience behind it.
Call The Big Question a 'piece of puff' if you like, but it seems to be quite good at piquing the interest of people on the forums, at least...
(Sorry this response is so delayed, by the way: I had to get my daughter to her saxophone lesson, walk the dog and watch Jules Holland, so I've only just caught up with the thread)
Seems to me that you want to have your cake and eat it! You admit that your tests aren't objective but then go on about how wonderfully experienced your reviewers are, how consistent the methods are, etc. Why bother with all that when the tests are purely subjective?
Chris
I was responding to the specific suggestion we should do a BQ feature in which we change nothing for each listen, and then report any differences the participants say they heard.
Why bother with all that when the tests are purely subjective?
Chris
Because, in order to come up with an opinion that is worth anything to anybody, one needs a huge amount of experience with all types of kit in order to arrive at a conclusion that something is good, poor, or indeed brilliant.
Without that experience, and the ability to look at equipment side by side in a consistent environment, any subjective opinion loses value, of course. We are able to review a lot more equipment than most people could hope to track down in-store. So we need our subjective opinion to be one that people respect and can then use to make a buying decision.
I'm not sure how that is having my cake and eating it. Being consistent isn't the same as being 'scientific'.
I have an idea: my team of monkeys are cheap, available at short notice and pretty good at telling their Justin Bieber from their Johannes Brahms.
Plus, if they dont like somehting, they throw poo at it.
Which, I'm sure you would agree, is very in tune with the Great British public.
I have an idea: my team of monkeys are cheap, available at short notice and pretty good at telling their Justin Bieber from their Johannes Brahms.
Plus, if they dont like somehting, they throw poo at it.
Which, I'm sure you would agree, is very in tune with the Great British public.
What was your idea?
Seems to me that you want to have your cake and eat it! You admit that your tests aren't objective but then go on about how wonderfully experienced your reviewers are, how consistent the methods are, etc. Why bother with all that when the tests are purely subjective?
I just don't get this question.
In the world of wines, many wine lovers will read reviews of wines from connoisseurs - people who have tried many, many wines and are therefore able to give their own, subjective opinion of other wines based on this experience.
In the world of video games, video gamers will read reviews from professional game reviewers - people who have played many, many games of each genre and are therefore able to give their own, subjective opinion of other games based on this experience.
In the world of fishing, many fishers would... etc. etc.
None of these would be requested to post double blind, scientific tests to prove their subjective opinions as objective facts. Why would hi-fi / home cinema reviewing be different in this respect?
You have all wasted your time.
Oh look...
You have all wasted your time.
Oh look...
Wrestling?! 
etc. etc.
I don't get it, I would like some more examples please. (Preferably something akin to the guest publication from the missing words round.)
You have all wasted your time.
Oh look...
Wrestling?!
I'm lost.
etc. etc.
I don't get it, I would like some more examples please. (Preferably something akin to the guest publication from the missing words round.)
etc. etc.
I don't get it, I would like some more examples please. (Preferably something akin to the guest publication from the missing words round.)
I bet they have all sorts of debates on What Water Tower Water & Concrete Forums on whether water towers should be assessed objectively or subjectively.
- Login to post comments
Jolly good: as both I and my colleague have said, all are welcome to apply. But really we have better things to do than invite readers in for these events at their own expense, then make them look daft in print.
Why would anybody look daft? Blind tests would answer the questions posed, and both the participants and readers would benefit from knowing whether any differences perceived were imagined or real. Surely nobody would object to that?
Was about to say exactly that. Well said!
Chris