I'm sorry you feel that - that would be the outcome, I genuinly don't and its a test that at some point will have to be done to justify the other test you do if for no other reason than to give you and your readers a point of reference of how the mind can affect what we believe we hear.
But from one persons feedback on what a day at WHF hq is like I still think most would consider it a fine day out, I mean honestly, you give them biscuits....for free!
And all that would achieve would be to make the reader-participants look foolish, and discourage others from offering their services for the feature.
I agree with Thompsonuxb. This would make an interesting test and good starting point for an interesting discussion topic.
I don't think that it would make the participants look foolish or discourage others from offering their services for the future either.
PC > AVI Neutron Five 2.1
Sony NWZ-A847 64GB Walkman > Westone UM3x
'Have to be done'? I really don't think so...
HiFi / A/V / Bedroom
I read the first 2 pages, quickly got bored, skipped to P5 and am shocked it's become yet another "WHF should behave like New Scientist" thread. Some of you guys are completely missing the point of music. I know it's fun to debate the pedantic detail between us, but to expect a leisure magazine such as WHF to take some these quadrupal independant third party statistically validated blind test challenges seriously, let alone the average reader being interested! Unless of course some of you are the editorial staff, and it's a double bluff.
JRiver MC17 -> Cambridge Audio DACmagic+ -> Roksan Caspian M2 -> ProAc D18
My Linn DS sounds noticeably better with Black Ravioli under the chassis!
I'm significantly better with some cooked ravioli in mine!
Onkyo TX-NR818 / Tannoy Revolution DC4 (bi-amped)
AVI Laboratory Series CD Player
"Everything has been said before, but since nobody listens we have to keep going back and beginning all over again." André Gide
Musical fidelity M6i Roksan caspian M2 cdp Atc scm 40s Tellurium Q black speaker cable chord chamelian xlr graham slee novo sennheiser hd598
So it's just a piece of "puff" then! The problem is that there are doubtless some of your readers who believe what they read and possibly spend a lot of their hard-earned money on that basis. Might I suggest that you owe your readers a bit more than that!
My cynical side says that there is no incentive for the industry (and that includes the magazines) to find definitive answers because they would have less to sell / write about. Maybe I'm being too harsh but ...
PS I'd be happy to take part in a listening test but only if it were scientifically conducted, ie double blind and statistically significant.
If those are your conditions, Chris, I fear we won't be seeing you down here. That is not the point of these days, as I explained. (Although you're still very welcome to come and see us for a BQ - I'm sure you'd find it interesting to see how we work.)
You'll be aware, no doubt, that WHF has never done testing like that. As has been mentioned or at least hinted at a couple of times on this thread, what we try to do is give an informed and experienced opinion on the equipment we review, in as real-world a way as possible. And through that help people come up with a shortlist of likely products that will suit them.
Our team has many, many years of cumulative experience behind them, and we are as consistent as we can be in testing all the kit that comes through our doors: everything is reviewed in-house by our full-time team; everything is looked at by at least two, and often more, members of the team; all reviews are then checked to ensure that what's written up matches what was decided at the reviewing stage. That way we have at least some system of checks and balances to allow for the frailties of human nature.
But no, it's not bench tested at any stage.
So yes, ours is a subjective rather than an objective opinion. But it's one with decades of experience behind it.
Call The Big Question a 'piece of puff' if you like, but it seems to be quite good at piquing the interest of people on the forums, at least...
(Sorry this response is so delayed, by the way: I had to get my daughter to her saxophone lesson, walk the dog and watch Jules Holland, so I've only just caught up with the thread)
Jonathan Evans is managing editor of What Hi-Fi? Sound and Vision
What, from here?
With these feet?
No one is asking WHF to become a science journal but It would nice to see just one 'Big Question' approached using a more rigorous double blind approach. I would be quite happy to take part and would not feel daft if I heard differences between identical systems.
This has been debated many times on the forum. Why not settle the issue by giving rigorous testing a go then we can all move on...what is there to lose?
Sonos Connect, Cyrus DAC XP Signature with PSX-R
Cyrus Mono X 300 Signature, Wilmslow Phoenix with Bass Extenders.
I think the problem being that some people will never ever " move on " regardless of the outcome of any tests.
Personally I made a decision to only use basic accessories. Amp on a shelf, bog standard USB cable, multi strand generic speaker cable. Altering these components may or may not make a difference. I believe the differences they make are probably very small say compared to room acoustics, speaker choice, record or CD ..e.t.c.
I'm with Jonathan on this, and like several others here I had a very enjoyable day at WHF Towers last year at a TBQ session, so I 'get' where the reviewers are coming from. Science has no place in listening at home, any more than it does at a gig or in a concert hall. It can be enlightening, but is no subsitute for simply listening.
Many moons ago I worked in a hi-fi shop as a 'Saturday boy', for over a decade up to the launch of CD. There is no doubt in my mind that you can 'test' things 'til you are blue in the face bit nothing beats experience and familiarity. My old boss, the store manager, could tell if I had changed the cables from outside the dem room door, and we all got to know the sound of the brands we sold most: NAD, Leak, Quad, Harman/Kardon, Dual, KEF, Tandberg, etc.
Cables and tables were not so faddish then but they certainly made a difference, as did basics like speaker stands and proper positioning, not to say setting up turntables properly (most people's prime source before CD). That is why a dealer is still a good guide, along with reviews, but only you can decide what you like. And in my experience, pretty much everything makes a difference in audio - it is mostly a matter of deciding if you like it or not.
Hi-Fi: Krell KAV-300cd, Michell TecnoDec/RB250/Grado Prestige Black1, KAV-300i amp, Transparent balanced interconnects and bi-wire to Sonus faber Concerto grand piano speakers, Nakamichi ZX-7 cassette deck, Logitech Squeezebox Touch, Hitachi FT-5500 and Sony S570ES tuners, BCD Engineering stand, RA Powerlink, Chord powerchord, Grado SR60i cans.
AV: Sony Bravia KDL-32EX503 telly, BDP-S370 player with QED HDMI. Currently unused: Denon AVR-1705, DVD-1710, KEF KHT1005.2
Science has no place in listening at home, any more than it does at a gig or in a concert hall. It can be enlightening, but is no subsitute for simply listening.
BRB, going get up and have a boogey to how my speakers measure
TBH, the whole style of what hifi doesn't really get into the scientific side at all, nor does it pretend to be anything other than subjective reviews by experiences reviewers. Making the big question sessions more scientifically rigorous would kinf of defeat the purpose of them which is, to my understanding, a bit of fun and insight, although I'm sure that some readers, or at least some forumites, wouldn't mind that.
Observe the signature in its natural habitat.
What would be lost to the magazine could be huge. IF the test proved that people really thought that there were differences when only a single system were played 3 times then subjective 'testing' should be dead in the water. Why would anyone then purchase a 'subjective testing' magazine to aid their interest and purchases?
To go back a few pages, I think you can over-estimate the intelligence of the general public. How many times have we read about people putting together a system made up of '5 star' products and being thoroughly unhappy with the sound that it produces?
Apple Lossless - ATV3 - AVI ADM 40 also ATV3 into AVI ADM 9T [my wife's system]
and Grado SR80i
You have answered your own question...
I should say I don't agree with your question or your answer.
Anyhow, WHF? staff have already told us many times - even in this thread - that their reviews are subjective so it's pointless devising a test to prove it!
Marantz M-CR603 + AirPlay • Rega R3 loudspeakers • iPhone 5 • iMac • Apple Airport Extreme 802.11n • Apple iPad Mini • Panasonic TX-L32D25B • Sony BDP-S390 • Ruark Audio R1 Deluxe • Humax HDR-Fox T2
Why all the talk of science? You don't need a PHD, a laboratory and white coats to do a blind test, hell, the big question that's in question was 99% there, all was needed was to ask the participants to pick out whatever configuarations they had said sounded different.
The reasons for not doing so are nothing to do with complexity, and doing so certainly wouldn't be boring or make anybody look daft, in fact it's daft to suggest so!!
© 2013 Haymarket Publishing