Tuesday, October 30, 2007

People who can think: 1. WHO: 0

Agoraphilia has an outstanding explanation of errors in 'scoring', when it comes to the WHO and ranking health care.
Nevertheless, the resulting composite scores are not sensible. Because the five component factors were measured on different scales to begin with, the WHO researchers had no choice but to scale them to make them comparable. When they scaled them, they used the approach described above: they measured a country’s factor score as the percentage of the distance between the lowest-scoring and highest-scoring countries for that factor.

As a result, a factor could have an exaggerated effect on the composite health performance scores if the raw scores for that factor were bunched more tightly than were other factors. If, for instance, if financial fairness ranged from 0.5 to 10.0 on the fairness scale, countries with fairness of 0.5 would be treated as having a fairness of 0. Essentially, a country that is somewhat fair would get treated as not fair at all. (This is assuming the raw fairness measure is meaningful to begin with, which I suspect it is not.)

It doesn't take much thinking to recognize this. Apply this thinking to a number of issues, and you'll find lots and lots of meaningless or questionable conclusions.

No comments: