436 posts / 0 new
Last post
Anonymous
Anonymous's picture
RE: more 'snake oil'

Why all the talk of science? You don't need a PHD, a laboratory and white coats to do a blind test, hell, the big question that's in question was 99% there, all was needed was to ask the participants to pick out whatever configuarations they had said sounded different.

The reasons for not doing so are nothing to do with complexity, and doing so certainly wouldn't be boring or make anybody look daft, in fact it's daft to suggest so!!

Covenanter's picture
Offline
Last seen: 14 hours 16 min ago
Joined: 20/07/2012 - 10:16
Posts: 1350
RE: more 'snake oil'

Dynamight wrote:

Andrew Everard wrote:

Thompsonuxb wrote:
I'd be happy to take part, I'm gonna email your mate.

Jolly good: as both I and my colleague have said, all are welcome to apply. But really we have better things to do than invite readers in for these events at their own expense, then make them look daft in print.

 

Why would anybody look daft? Blind tests would answer the questions posed, and both the participants and readers would benefit from knowing whether any differences perceived were imagined or real. Surely nobody would object to that?

 

Was about to say exactly that.  Well said! 

Chris

Marantz PM8005 / SA8005 / KEF R700s / AKG K702

relocated's picture
Offline
Last seen: 3 hours 16 min ago
Joined: 20/01/2012 - 12:40
Posts: 1108
RE: more 'snake oil'

chebby wrote:

relocated wrote:
Why would anyone then purchase a 'subjective testing' magazine to aid their interest and purchases?

You have answered your own question...

relocated wrote:
...I think you can over-estimate the intelligence of the general public.

I should say I don't agree with your question or your answer.

Anyhow, WHF? staff have already told us many times - even in this thread - that their reviews are subjective so it's pointless devising a test to prove it!

 

Well Chebby if you have read my post and come up with that response then I have clearly over-estimated your IQ.   :wave:

Apple lossless - Netgear Nighthawk - ATV3 - AVI ADM 40.  

AVI ADM 9T used in my wife's system

Covenanter's picture
Offline
Last seen: 14 hours 16 min ago
Joined: 20/07/2012 - 10:16
Posts: 1350
RE: more 'snake oil' RE: more 'snake oil'

Jonathan_E wrote:

Covenanter wrote:

So it's just a piece of "puff" then!  The problem is that there are doubtless some of your readers who believe what they read and possibly spend a lot of their hard-earned money on that basis.  Might I suggest that you owe your readers a bit more than that!

My cynical side says that there is no incentive for the industry (and that includes the magazines) to find definitive answers because they would have less to sell / write about.  Maybe I'm being too harsh but ...

Chris

PS I'd be happy to take part in a listening test but only if it were scientifically conducted, ie double blind and statistically significant.

If those are your conditions, Chris, I fear we won't be seeing you down here. That is not the point of these days, as I explained.  (Although you're still very welcome to come and see us for a BQ - I'm sure you'd find it interesting to see how we work.)

You'll be aware, no doubt, that WHF has never done testing like that. As has been mentioned or at least hinted at a couple of times on this thread, what we try to do is give an informed and experienced opinion on the equipment we review, in as real-world a way as possible. And through that help people come up with a shortlist of likely products that will suit them.

Our team has many, many years of cumulative experience behind them, and we are as consistent as we can be in testing all the kit that comes through our doors: everything is reviewed in-house by our full-time team; everything is looked at by at least two, and often more, members of the team; all reviews are then checked to ensure that what's written up matches what was decided at the reviewing stage. That way we have at least some system of checks and balances to allow for the frailties of human nature.

But no, it's not bench tested at any stage.

So yes, ours is a subjective rather than an objective opinion. But it's one with decades of experience behind it. 

Call The Big Question a 'piece of puff' if you like, but it seems to be quite good at piquing the interest of people on the forums, at least... 

 

(Sorry this response is so delayed, by the way: I had to get my daughter to her saxophone lesson, walk the dog and watch Jules Holland, so I've only just caught up with the thread)

Seems to me that you want to have your cake and eat it!  You admit that your tests aren't objective but then go on about how wonderfully experienced your reviewers are, how consistent the methods are, etc.  Why bother with all that when the tests are purely subjective?

Chris

Marantz PM8005 / SA8005 / KEF R700s / AKG K702

Andrew Everard's picture
Offline
Last seen: 1 year 3 weeks ago
Joined: 30/05/2007 - 12:34
Posts: 28836
RE: more 'snake oil'

Dynamight wrote:
The reasons for not doing so are nothing to do with complexity, and doing so certainly wouldn't be boring or make anybody look daft, in fact it's daft to suggest so!!

I was responding to the specific suggestion we should do a BQ feature in which we change nothing for each listen, and then report any differences the participants say they heard.

Jonathan Evans's picture
Offline
Last seen: 5 days 7 hours ago
Joined: 07/01/2008 - 16:15
Posts: 211
More 'snake oil'

Covenanter wrote:

Why bother with all that when the tests are purely subjective?

Chris

 

Because, in order to come up with an opinion that is worth anything to anybody, one needs a huge amount of experience with all types of kit in order to arrive at a conclusion that something is good, poor, or indeed brilliant.

Without that experience, and the ability to look at equipment side by side in a consistent environment, any subjective opinion loses value, of course.  We are able to review a lot more equipment than most people could hope to track down in-store. So we need our subjective opinion to be one that people respect and can then use to make a buying decision.

I'm not sure how that is having my cake and eating it. Being consistent isn't the same as being 'scientific'.

Managing editor, What Hi-Fi? Sound and Vision

Anonymous
Anonymous's picture
RE: more 'snake oil'

I have an idea: my team of monkeys are cheap, available at short notice and pretty good at telling their Justin Bieber from their Johannes Brahms.

Plus, if they dont like somehting, they throw poo at it.

Which, I'm sure you would agree, is very in tune with the Great British public.

 

BenLaw's picture
Offline
Last seen: 9 hours 6 min ago
Joined: 21/11/2010 - 20:21
Posts: 6394
RE: more 'snake oil'

Biggerboat wrote:

I have an idea: my team of monkeys are cheap, available at short notice and pretty good at telling their Justin Bieber from their Johannes Brahms.

Plus, if they dont like somehting, they throw poo at it.

Which, I'm sure you would agree, is very in tune with the Great British public.

 

 

What was your idea?

professorhat's picture
Offline
Last seen: 2 days 22 hours ago
Joined: 28/12/2007 - 11:34
Posts: 11029
RE: more 'snake oil' RE: more 'snake oil'

Covenanter wrote:

Seems to me that you want to have your cake and eat it!  You admit that your tests aren't objective but then go on about how wonderfully experienced your reviewers are, how consistent the methods are, etc.  Why bother with all that when the tests are purely subjective?

I just don't get this question.

In the world of wines, many wine lovers will read reviews of wines from connoisseurs - people who have tried many, many wines and are therefore able to give their own, subjective opinion of other wines based on this experience.

In the world of video games, video gamers will read reviews from professional game reviewers - people who have played many, many games of each genre and are therefore able to give their own, subjective opinion of other games based on this experience.

In the world of fishing, many fishers would... etc. etc.

None of these would be requested to post double blind, scientific tests to prove their subjective opinions as objective facts. Why would hi-fi / home cinema reviewing be different in this respect?

 

The owls are not what they seem...

Alec's picture
Offline
Last seen: 1 day 9 hours ago
Joined: 08/10/2007 - 21:06
Posts: 6114
RE: more 'snake oil' RE: more 'snake oil'

You have all wasted your time.

Oh look...

professorhat's picture
Offline
Last seen: 2 days 22 hours ago
Joined: 28/12/2007 - 11:34
Posts: 11029
RE: more 'snake oil' RE: more 'snake oil'

Alec wrote:

You have all wasted your time.

Oh look...

Wrestling?! :grin:

 

The owls are not what they seem...

BenLaw's picture
Offline
Last seen: 9 hours 6 min ago
Joined: 21/11/2010 - 20:21
Posts: 6394
RE: more 'snake oil' RE: more 'snake oil'

professorhat wrote:

etc. etc.

 

I don't get it, I would like some more examples please. (Preferably something akin to the guest publication from the missing words round.)

Alec's picture
Offline
Last seen: 1 day 9 hours ago
Joined: 08/10/2007 - 21:06
Posts: 6114
RE: more 'snake oil' RE: more 'snake oil'
Alec's picture
Offline
Last seen: 1 day 9 hours ago
Joined: 08/10/2007 - 21:06
Posts: 6114
RE: more 'snake oil' RE: more 'snake oil'

professorhat wrote:

Alec wrote:

You have all wasted your time.

Oh look...

Wrestling?! :grin:

 

I'm lost.

professorhat's picture
Offline
Last seen: 2 days 22 hours ago
Joined: 28/12/2007 - 11:34
Posts: 11029
RE: more 'snake oil' RE: more 'snake oil'

BenLaw wrote:

professorhat wrote:

etc. etc.

I don't get it, I would like some more examples please. (Preferably something akin to the guest publication from the missing words round.)

 

The owls are not what they seem...

Pages

Log in or register to post comments