Doctors are the first line of defense in ensuring women receive sound advice about a new generation of prenatal screens and other complex tests. But many health professionals’ muddled understanding of statistics may have serious consequences for their patients.
An investigation last month by the New England Center for Investigative Reporting, for instance, found that the four major companies offering prenatal tests oversell the accuracy of their screens and that some expecting women, swayed by impressive statistics and shaky advice from their doctors, have aborted healthy fetuses based on false positives.
Just how much trouble some doctors have interpreting a positive screening result is demonstrated in a study – “Medicine’s Uncomfortable Relationship With Math” – published in the June 2014 issue of JAMA Internal Medicine. When asked to calculate the probability that a positive screen result was actually correct, the majority of doctors surveyed at one Boston hospital got that number wrong.
Researchers at Harvard and MIT asked a mix of 61 medical students and doctors to determine a critical but confusing figure known as the positive predictive value in a hypothetical medical test. More than three quarters of those tested – 47 of 61 – gave the wrong answer. The positive predictive value tells doctors the probability that a patient who tested positive for a certain disease truly has that disease. Proper grasp of this value is critical when it comes to deciding what to do when a test result comes back positive. The more rare the condition, the higher the rate of false alarms.
Correctly determining PPV requires doctors to trudge through some serious statistical weeds. The doctors surveyed consistently overestimated the accuracy of the test, a tendency that could significantly impact diagnosis and treatment of patients.
“Statistics is under-emphasized in medical education,” said Arjun Manrai, the lead author of the study, who is a PhD candidate at the Harvard-MIT Division of Health Sciences and Technology. “Probability and statistics can be counter-intuitive and require ample time for training and practice, just like other aspects of medicine.”
With the explosion of ever more specialized medical tests, even doctors who have a good grip on the statistics of disease screening find themselves pushing against an avalanche of marketing statistics from the companies that put out these tests. For instance, while some of the prenatal testing companies pinpointed in NECIR’s investigation advertise accuracy rates above 99 percent, they gloss over the probability that a positive screening result is actually correct – a probability which some studies have shown to be less than 50 percent for many conditions.
Manrai believes that as more new medical tests enter the market, doctors must become more attuned to the complicated statistics behind a test’s reliability.
“Many doctors have an intuitive understanding of the tests that they commonly order,” he said. “But as care becomes more personalized, new and unfamiliar tests are going to be developed. This will make improving statistical literacy even more crucial.”