The Apixio Blog

Mind the Gap: Where Evidence-Based Medicine Fails

Evidence-based medicine (EBM) is a popular approach to helping physicians make appropriate care decisions. It has gained tremendous popularity within the medical community; in a 2013 survey, 75% of health system chief medical officers said that EBM had gained some acceptance in their organization. EBM can be a very valuable way to provide treatment guidance to physicians based upon medical literature. But there are some potentially serious shortfalls in its application.

The EBM method popularized in the late 1980s reacted to decades of research demonstrating that medical decision making was prone to cognitive bias and that consensus expert opinion was inconsistently followed, leading to inappropriate care. EBM curates data from tens or hundreds of studies in an attempt to rank disease treatments and present care recommendations for physicians.

Not all research is treated equally in EBM. There are different methods used to conduct medical research, and each has different power, or strength, to demonstrate that a treatment can actually impact a patient’s disease, all other things being equal.

For example, when determining whether a drug can reduce cholesterol in a population compared with a placebo (sugar pill), researchers need to control for other habits, therapies, or diseases associated with abnormal cholesterol levels. By controlling for other factors which influence the intended outcomes researchers establish cause and effect.

The “gold standard” study design (and strongest EBM recommendation) is the randomized controlled trial (RCT). In our example of the cholesterol drug, individuals in the study would be randomly selected to receive either the intervention or placebo.  That way, other factors associated with cholesterol levels in the blooddiet, exercise, medications, conditionsare very likely to be equally present among the group taking the drug and the placebo. The effect of the drug can then be isolated. Studies based on non-RCT methodologies (e.g. case-control study, observational study, systematic review) cannot show cause and effect as strongly.

One problem with applying EBM to everyday practice is that most studies aren’t conducted as a RCT. Each year, each major medical specialty society (e.g. American College of Cardiology) publishes EBM guidelines for the treatment of specific diseases such as coronary heart disease or atrial fibrillation, a disease affecting the electrical activity of the heart.  Only 11% of the ACC guidelines are based upon multiple RCT studies, which get a “Level A” rating. The remainder of the EBM guidelines are based upon weaker methodologies and therefore cannot get the strongest recommendation.

Increasingly, physicians cannot keep straight which EBM recommendations within a medical specialty are the strongest and which ones are not. And increasingly, with the large number of studies and guidelines and recommendations, all EBM tends to be treated similarly.

For example, in 1992, the American College of Physicians made an EBM recommendation that women take estrogen replacement postmenopause, based on case-control studies indicating that estrogen reduced the symptoms of menopause. Because these studies were considered “evidence-based”, millions of women and their physicians followed the recommendation.

Years later, the “evidence-based” recommendation (or guideline) was reversed given newer studies which showed that estrogen could increase the risk of breast cancer, coronary heart disease, and stroke. In this case, adherence to EBM at that time may have actually worsened care.

The main problem here is not with EBM per se. It is that it is too expensive and time consuming to conduct RCT for each and every medical treatment or intervention. So the basis for much of “evidenced based” guidelines is weak and prone to being refuted, as was the case with estrogen use.  

Even when a RCT is done (grade A evidence), it tends to have relatively small study groups which do not represent a larger, more diverse population. For example, while the cholesterol-lowering drug may work for the young man from Ireland in a RCT, it may not be as good for the older lady from Southern China, where this group was not represented in the RCT. A physician in Canton, China prescribing the cholesterol medication based upon a RCT in which there were no similar study patients from the region may not be giving the optimal treatment. So researchers rely on weaker, but less expensive and time consuming, study methods to establish the efficacy of a medical treatment across many diverse populations.  

With the greater adoption of electronic medical records, there are now ways to study a very large group of individuals and apply methods that very closely mimic a RCT. In this regard, we can study a greater number of treatments for their efficacy and establish a greater body of solid literature on what does and does not work with the construct of EBM.

Tens of millions of clinical records are generated and updated daily, which provides a treasure trove of knowledge about clinical medicine and healthcare delivery. With statistical techniques that can assemble individuals into cohorts, we can answer questions about what treatment work best in the care of specific diseases in specific populations in “real world” settings.

This technique, only possible by leveraging powerful computers trained to abstract information from large sets of records, has been called, practice-based evidence (PBE). PBE is a powerful method towards personalizing recommendations to a patient based upon analysis of tens of thousands of similar patients.
Keep following our blog to find out more about PBE.

© Apixio 2015