Real World Evidence Standards: An Industry Call To Action

There is a plethora of important data in RWE. If we intend to use this data to support promotional claims, we must collectively, working with all involved parties, create standards to which we hold ourselves accountable. This is an imperative. Otherwise, the unintended consequences become a nightmare for all involved; our patients, their caregivers, their physicians, and for our business. We must do this to ensure use of this data is competent and reliable, benefits patients, is sustainable, minimizes litigation and Lanham Act suits, holds all companies to the same standards, and maintains industry credibility with all stakeholders.

This is hard, as there are a large number of challenging considerations to developing standards. Different groups are struggling to lead this. FDA is working daily to create regulatory standards. But this is also an industry opportunity. If we believe this data represents reliable information and that healthcare can rely on it, we must be willing to work to create robust, above reproach, data-driven standards to which we hold ourselves accountable. We have an opportunity to demonstrate our authentic commitment to advancing robust science and patient centricity. Now is the time.

Imagine a worse-case scenario. Imagine a mega database that can be evaluated in limitless ways. The company designs 200 exploratory searches to demonstrate drug A is better than drug B in patience adherence, tweaking different assumptions in each one. The programs run over night and the morning results show 5 scenarios that trend towards drug A outperforming drug B. The other 195 are either equivocal or show drug B better than drug A. That is unexpected, as the assumptions were deliberately chosen to give drug A the best possible results. However, as 5 of the scenarios showed drug A better, the company chooses to do further evaluation and eventually has the “best” study published in a reputable journal.

Clearly, this scenario raises numerous, important questions. What happens to all of the other data? No one outside the company will ever know that 95% of the studies came to a very different conclusion, nor that the scenarios were run based on selection-biased assumptions. How many patients might be harmed, either from lack of claimed effect, or delayed treatment with other proven products? How much unnecessary money will payers pay? What are the overall corporate compliance risks – product/payer litigation, DOJ investigation and subsequent fines, FDA enforcement, universal loss of credibility, loss of future research support, and damaged customer relationships, among many others.

No company representative reading this would ever condone such a scenario. But it illustrates some of the possibilities and reasons standards are an imperative. RWE standards are essential to ensure data confidence, optimal patient outcomes, and a “safe,” level playing field.

What should RWE standards address? I do not pretend to have all of the answers. The FDA is working daily on this. The Duke-Margolis Center for Health Policy is also leading this work, as are other groups. But, what is certain, is that we must find the will, and the leadership, to define and address those questions and create the standards by which we hold ourselves accountable. At a minimum, they must address both the data standards and definitions of the study design AND the appropriate context of presenting the information, with primary goals of producing scientifically sound data and minimizing bias, such as the following (NOT all inclusive):

1.    Study methodology and design adequacy, consistent with current, rigorous, scientific standards, designed to minimize bias

a.    Study intent/justification

b.    Areas of appropriate RWE usefulness

c.    Assumptions/endpoints selection appropriateness

d.    Data base (s) selection appropriateness and evaluation

e.    Inclusion criteria (patient demographics, disease severity, prior treatment, etc.)

f.     Data standard definitions (competent and reliable, for instance)

g.    Statistical considerations

h.    Control of variables

i.     Limitations of data

2.    Standards for data sharing must address the imperative for reducing the potential for bias, selection and otherwise

a.    Publication standards

                   i.    Study selection and transparency, including what is not published

                   ii.    Publication journal standards (or other publication type)

b.    Information dissemination – how, to whom and why

c.    Disclosure considerations

                     i.    Study design and limitations

                     ii.    Additional needed studies

  iii.    Relevant information from other studies, including differing conclusions

                    iv.    Author/researcher relationships with study supporter

                     v.    Inconsistencies with product labeling

In addition to the above minimal considerations, process and structure are imperatives to ensure consistency and accountability. For instance, as thought-starters:

1.    Should the standards also create a standardized template for presenting RWE, such as the one developed for sharing clinical trial information on CT.gov? The template could potentially help minimize discretion and variability in the application of the standards.

2.    How does this get memorialized and enforced? Should the industry, working with a broad group of stakeholders, such as Duke-Margolis and/or other nonprofit centers, create an industry code focused on RWE?

3.    Should the standards encourage the development of corporate internal processes, and perhaps provide examples, for ensuring that appropriate standards are applied to company communications involving RWE? For example, is additional rigorous scientific input required for RWE, even beyond the current standard promotional review committee structure?

Our industry, as one of the primary consumers of this data, has both a right, and a responsibility to help lead the development of the highest quality RWE standards. And, we have a responsibility to ensure the data is consistently of a scientifically sound standard and is developed and shared in an unbiased way. If we can’t create (and follow) consistent, data-driven standards - if it is too difficult - we must ask ourselves the obvious. Should we be using that data in promotion?