Have your say & ask the experts!

Question on the ratings

11 replies [Last post]
Muzza
Muzza's picture
Offline
Joined: 11 Apr 2011
Posts: 97

Hi

I have been reading the mag for ages but it strikes me that almost everything gets 3,4 or 5 stars.

This doesn't really give a huge spread and makes everything seem the same, so I was wondering if it would be better to make more use of the 1 and 2 stars or maybe make it out of 10 or maybe even a 100?

chebby
chebby's picture
Online
Joined: 2 Jun 2008
Posts: 13214
RE: Question on the ratings

Probably because WHF doesn't go out of it's way to seek outright 'tat' for review.

Having said that, I can see one and 2 star items when I search the reviews section.

__________________

Marantz M-CR603 + AirPlay • Rega R3 loudspeakers • iPhone 5 32GB • iMac • Apple Airport Extreme 802.11n • Panasonic TX-L32D25B • Sony BDP-S390 • Ruark Audio R1 Deluxe • Humax HDR-Fox T2

professorhat
professorhat's picture
Offline
Joined: 28 Dec 2007
Posts: 10255
RE: Question on the ratings

3 is average for the money. So it would be unfair to give a score of less than 3 if the device performed to this level would it not?

 

Lee H
Lee H's picture
Offline
Joined: 7 Oct 2010
Posts: 1462
RE: Question on the ratings

Muzza wrote:
Hi I have been reading the mag for ages but it strikes me that almost everything gets 3,4 or 5 stars. This doesn't really give a huge spread and makes everything seem the same, so I was wondering if it would be better to make more use of the 1 and 2 stars or maybe make it out of 10 or maybe even a 100?

 

It's a fair point, but you have to take the ratings together with the price. A very expensive product can get 3 stars because for the price it should have done better. Whereas a cheap 5 star product can be as much because it was suprisingly good as anything else. 

The star rating alone doesn't give the full picture

__________________

Formerly Still known as Lee H

Muzza
Muzza's picture
Offline
Joined: 11 Apr 2011
Posts: 97
RE: Question on the ratings

professorhat wrote:

3 is average for the money. So it would be unfair to give a score of less than 3 if the device performed to this level would it not?

 

Agreed but by this rationale most of the things WHF review is average or better than average for the money?

Muzza
Muzza's picture
Offline
Joined: 11 Apr 2011
Posts: 97
RE: Question on the ratings

Lee H wrote:

Muzza wrote:
Hi I have been reading the mag for ages but it strikes me that almost everything gets 3,4 or 5 stars. This doesn't really give a huge spread and makes everything seem the same, so I was wondering if it would be better to make more use of the 1 and 2 stars or maybe make it out of 10 or maybe even a 100?

 

It's a fair point, but you have to take the ratings together with the price. A very expensive product can get 3 stars because for the price it should have done better. Whereas a cheap 5 star product can be as much because it was suprisingly good as anything else. 

The star rating alone doesn't give the full picture

 

I agree but I jsut thought a larger spread of ratings would be more helpful for novices like me!

The_Lhc
The_Lhc's picture
Offline
Joined: 16 Oct 2008
Posts: 11138
RE: Question on the ratings

Muzza wrote:

professorhat wrote:

3 is average for the money. So it would be unfair to give a score of less than 3 if the device performed to this level would it not?

 

Agreed but by this rationale most of the things WHF review is average or better than average for the money?

Most things are average. That's why it's the average.

__________________

No signature worth mentioning...

professorhat
professorhat's picture
Offline
Joined: 28 Dec 2007
Posts: 10255
RE: Question on the ratings

Muzza wrote:

professorhat wrote:

3 is average for the money. So it would be unfair to give a score of less than 3 if the device performed to this level would it not?

Agreed but by this rationale most of the things WHF review is average or better than average for the money?

Indeed it does. My point was, if WHF review 10 products of the same type, in the same price bracket, and if none of them produce results which are not considered worse than average, what would be the justification for giving some 1 or 2 stars? Surely, they all have to get 3 stars or above?

They can only review and then report on what what they see or hear.

 

John Duncan
John Duncan's picture
Offline
Joined: 8 Jan 2008
Posts: 20436
RE: Question on the ratings

But then if 10 TVs picked at random are better than average, where did the average come from?

Aaaaaaaaaaanyway, this ignores the fact that the rating system doesn't say that 3 is average.  I forget what it does say, though...

EDIT - it means 'decent'

RE-EDIT - damn, it says 'average' in the buyer's guide.  I'll get me coat...

__________________

CA StreamMagic 6  |  CA 751BD  |  CA 651A  |  PMC DB1i

Moderator. mail: john.duncan.whf at the mail of g dot com

professorhat
professorhat's picture
Offline
Joined: 28 Dec 2007
Posts: 10255
RE: Question on the ratings

John Duncan wrote:

But then if 10 TVs picked at random are better than average, where did the average come from?

Aaaaaaaaaaanyway, this ignores the fact that the rating system doesn't say that 3 is average.  I forget what it does say, though...

EDIT - it means 'decent'

Okay, decent rather than average. Point remains the same though i.e. if none of the products are "Disappointing" (2 stars) or "Awful" (1 star), it would be unfair to rate them as such.

So the only solution would be to actively seek out products that were either disappointing or awful and then include them. But I'm not sure what the point of that would be.

John Duncan wrote:

RE-EDIT - damn, it says 'average' in the buyer's guide.  I'll get me coat...

Victory is mine! (it's a bad day, I'm relishing everything I get Wink )

 

Lee H
Lee H's picture
Offline
Joined: 7 Oct 2010
Posts: 1462
RE: Question on the ratings

professorhat wrote:

if none of them produce results which are not considered worse than average, what would be the justification for giving some 1 or 2 stars? Surely, they all have to get 3 stars or above?

 

Unlike our annual performance review at work where a department has to meet a ratings distribution curve. We can't all be good. When I was a people manager I was once asked which of my team I would be happy to downgrade to meet the curve!

__________________

Formerly Still known as Lee H

nopiano
nopiano's picture
Offline
Joined: 15 Feb 2009
Posts: 1000
RE: Question on the ratings

Yes, although 3 might supposedly mean "average" that doesn't mean literally that there as many 1 and 2 star products as 4 and 5.  I actually rather like JD's 'decent', or 'OK for the money, but could be better' which is what it seems to mean in practice.

@ Muzza, there have been debates about this before, but the 1 to 5 is easiest to pick up, and as others have said, needs to be qualified with "...for its price".  An absolute process with 100 meaning 'as good as it gets' or 'state of the art' would mean a DAB radio might score 12, which would make a new reader run a mile!  It would probably scare off advertisers too, and we'd have no mag to read.  But 4 stars means one of the better DAB radios for £80, which is what most buyers need to know. 

__________________

Hi-Fi: Krell KAV-300cd, Michell TecnoDec/RB250/Grado Prestige Black1, KAV-300i amp, Transparent balanced interconnects and bi-wire to Sonus faber Concerto grand piano speakers, Nakamichi ZX-7 cassette deck, Logitech Squeezebox Touch, Hitachi FT-5500 and Sony S570ES tuners, BCD Engineering stand, RA Powerlink, Chord powerchord, Grado SR60i cans.

AV: Sony Bravia KDL-32EX503 telly, BDP-S370 player with QED HDMI. Currently unused: Denon AVR-1705, DVD-1710, KEF KHT1005.2