Controversies in Management: Department of Health shoots itself in the hip Why the report of the Advisory Group on Osteoporosis undermines evidence based purchasing

Ploy to avoid considering costs and benefits

The advisory group was asked to “establish what information about osteoporosis is available, what research is being conducted and what further work needs to be done, and to report to ministers.” In making recommendations about the use of bone densitometry and about the number of machines the report goes well beyond this brief.

What is wrong with the advisory group's recommendations? Few would disagree that the 50% or so of hip fractures in older women associated with low bone density are a major cause of suffering and a cost to the NHS. Nor would many disagree that bone density measurement is a valuable diagnostic tool for clinicians treating patients with bone disease or that it is important for research. However, the value of bone density measurement to identify patients who could be encouraged to take preventive action (exercise, dietary change, and drug treatments such as hormone replacement therapy) against a fracture which they may or may not suffer more than 20 years in the future is highly uncertain.

The advisory group asserts that “population screening” for osteoporosis is not worth while but that bone densitometry should be used for testing patients likely to have low bone mass. They include in this group women at the time of the menopause in order to aid clinical decision making, and they say that 45% of women have “strong risk factors.” Yet if certain groups are known to have low bone density what is to be gained by testing them? If it is not known, then how are the benefits of testing this group any more certain than for the general population? The semantic sidestep whereby widespread testing of healthy people is described as “not screening” is nothing more than a ploy for circumventing the need to consider the overall potential benefits and costs of widespread testing. By accepting the report's recommendations, the Department of Health is undermining the firm national lead on the evaluation of potential new screening programmes promised by the chief medical officer.5

Value of densitometry in risk assessment

What is the evidence that bone density measurement is “essential for clinical decision making” for these groups? There are internationally accepted criteria for evaluating diagnostic health technologies and for reviewing publications on this topic.6 Two fundamental conditions must hold: the test must accurately predict patients who could potentially benefit (those who will have a fracture in the future) and there must be a cost effective treatment. Bone densitometry for risk assessment has problems on both counts.

Estimates from a recent systematic review of cohort studies—most of which as yet have only short term follow up—show bone density as rather a poor predictor, with a sensitivity of 38% and specificity of 88% (for 1 standard deviation below the mean cut off) and 9% sensitivity and 99% specificity (for a 2 standard deviation cut off).7 Thus even with the cut off for higher specificity (which misses 91% of potential fracture patients) only 56% of people so identified and recommended therapy will in fact be destined to sustain a fracture. In the context of this evidence the advisory group's assertion that bone densitometry is “the best tool available” and that it “provides specific and reasonably sensitive indices of fracture risk” is vague and misleading.

On the second count the report gives little systematic consideration to the nature and quality of the evidence for the effectiveness of interventions. There are, for example, no studies showing that use of bone densitometry in high risk groups results in a reduced incidence of fractures. There are also doubts about the persistence of any protective effect of hormone replacement therapy with age8 and after treatment is stopped.9

Other fundamental issues—important whether whole population or selective screening is under consideration—are barely considered in the report. These include the requirement for rigorous common standards of analytical performance to ensure that a test result would be the same independently of where or by whom the test is performed, the reference mean and standard deviations of bone mass to be used in British populations, and the cut off points. In Aberdeen the criteria used resulted in treatment being recommended for 50% of women at the time of the menopause; in other places the equivalent figures are 25% or less.10 Bone densitometry is also recommended in the report for monitoring therapeutic response. However, because precision of the measurement is low relative to the rate of bone loss, measurements need to be several years apart to be useful in detecting whether treatment was retarding bone loss.

The introduction of a new diagnostic health technology, particularly for prevention, is a complex issue which requires detailed assessment. The advisory group usefully highlights an important research agenda. However, because of the emphasis on bone density testing these aspects were lost on the media. National television news highlighted the fact that other countries invest more in dual x ray absorptiometry machines, and the General Practitioner carried the headline, “Officials now favour mass bone screening.” Health authorities faced already with funding an alarming list of effective interventions are not likely to find this report particularly constructive. They might be forgiven for wondering whose interest will best be served by recommending the increased purchase of bone densitometry and this back door advocacy of screening—the general public or the equipment and pharmaceutical suppliers?