Leaders at MGB are always in search of tools to improve care quality, minimize variation, and better demonstrate the value of care that they provide to their patients.
MGB is an integrated health system, which includes two academic medical centers, three specialty hospitals, seven community hospitals, a health plan, and a primary care network of more than 6,000 physicians. To date, MGB has one of the largest ePRO programs in the world, assigns more than 110,000 questionnaires per month, and has collected more than 14 million questionnaires.
Recognizing that ePROs positively impact patient care and outcomes (e.g., patient’s quality of life, time on treatment, and survival), MGB leadership provided top-down support to create a PROMs department to implement the use of ePROs across specialties, including oncology. Individual and aggregated ePROs are used to inform the patient-provider interaction.
Since implementing the PROMs program, MGB has seen sweeping uptake of the collection of ePROs used in patient care, with over 60 specialties using ePROs to care for patients. Additionally, MGB has leveraged aggregated ePRO data to identify care gaps for improvement at the provider, institution, and system levels. More broadly, the PROMs program has become a tool for MGB to support contracting and to inform pay-for-performance metrics.
MGB categorizes the impacts of their PROMs program into four main categories, which they call the "4 C's":
MGB’s “4 C’s” to illustrate PROMs program impact
The routine collection of ePROs in MGB’s clinics has led to improved clinician satisfaction, improved understanding of diverse disease processes,1 and has revealed structural inequities in how PROMs are collected.
MGB’s PROMs program improves benchmarking for quality outcomes across clinics, physicians, and patients. These comparative analyses help MGB address root causes to outlier care outcomes.
MGB uses ePROs to convince external stakeholders of the value of an intervention. MGB partners with pharmaceutical companies to capture quality of life and survival data for new cancer drug therapies.
MGB uses ePROs to inform pay-for performance contracts with payer organizations. MGB is reimbursed for ePRO data collection in private payer contracts and through its Medicaid ACO.
Conversations focused on ePROs are likely related to ongoing conversations around care management and remote patient monitoring. To incentivize and finance ePRO programs, leaders should consider them as part of the larger infrastructure investments required to support the transition to value-based care and care at home.
At Pfizer Oncology, we are committed to advancing medicines wherever we believe we can make a meaningful difference in the lives of people living with cancer. Today, we have an industry-leading portfolio of 24 approved innovative cancer medicines and biosimilars across more than 30 indications, including breast, genitourinary, colorectal, blood and lung cancers, as well as melanoma.
This report is sponsored by Pfizer, an Advisory Board member organization. Representatives of Pfizer helped select the topics and issues addressed. Advisory Board experts w rote the report, maintained final editorial approval, and conducted the underlying research independently and objectively. Advisory Board does not endorse any company, organization, product or brand mentioned herein.
For organizations looking to leverage ePRO data to support the point of care and drive improvement in overall quality and outcomes, below are MGB’s five keys to success
MGB’s quality and patient experience leaders recognized that spending money on infrastructure for an ePRO program would be futile if the organization didn’t first invest in a culture that would accept the program. Consequently, MGB intentionally invested in the following three factors while developing their program.
Factor 1: Buy-in
Leaders went department by department to identify physician champions for the PROMs program. Physician champions helped to create buy-in among their peers and quicken the pace of the program’s rollout. For example, departments with strong physician champions stood up the program within a few months, whereas departments without champions took a few years to implement the program. As part of this effort, leaders educated physicians about how ePRO data collection improves patient outcomes and how the data can support physicians during the patient encounter. Leaders also emphasized to physicians that the program would not significantly impact physician workflows.
Sample talking points for building buy-in among physicians:
Factor 2: Staffing
Simultaneously, leaders created a dedicated team of individuals to roll out and support the program. MGB invested in building a PROMs department, which includes program managers, content specialists, EHR analysts, and clinic support staff. Today, MGB has almost 10 FTEs to support their enterprise-wide PROMs initiative that spans 90 specialties and 213 clinics. This central team is responsible for building patient questionnaires, managing the data warehouse, and training and supporting providers and staff.
Advice from MGB
Programs need to first invest in people and technology to then build the infrastructure. You need to have dedicated FTEs and physician champions who can get providers on board and continually show providers how to use the data patients are providing.
Dr. Nadine Jackson McCleary
Mass General Brigham
Factor 3: Technology
MGB invested in the necessary infrastructure (technology, internet, etc.) for their program. MGB worked with their EMR vendor to operationalize workflows, coordinated with IT and tablet manufacturers, and upgraded their WIFI in many parts of the institution to support in-office tablet distribution.
With these three components in place (organizational buy-in, dedicated staff, and the required technology), MGB then rolled out the program across each specialty that had chair and practice manager signoff as well as physician engagement.
To be successful, an ePRO program must balance provider and patient ease of use and clinical utility. Here are some of MGB’s guiding principles for patient questionnaires to achieve that outcome.
Just because you can ask doesn’t mean you should
MGB’s questionnaires are validated instruments, psychometrically tested, andassociated with improved patient outcomes. Consequently, every question mustbe clinically meaningful and tied to an improvement in outcomes. This meansthat just because a question is interesting doesn’t mean it should be asked.
Ask questions when they clinically matter
MGB first assesses when a patient would expect to see a clinical improvement ordeterioration, and then sends questionnaires at or before that point. The ePROdata collected then allows for benchmarking of a patient’s subsequent statusagainst a patient’s initial status. Providers can then gauge improvement over timefrom the point of intervention or treatment.
Respect patients’ time
Health is only one facet of patients’ lives, so MGB intentionally accounts for patient burden and limits the number of questions patients are being asked. MGB questionnaires do not exceed 30 questions.
With these patient questionnaire principles in mind, MGB implemented the belowworkflow to execute their PROMs program. Patients fill out simple, well-timed questionnaires via the patient portal at home or via tablets in the patient waiting room before their appointment. Providers can then access patient responses during the visit via the EMR to inform the patient encounter.
Traditionally, ePRO programs collect an individual patient’s data to inform that patient’s care. MGB takes this model a step further and provides physicians with both individual and aggregated ePRO data, which provides context to support clinician decision-making and conversations with patients. Physicians can harness ePRO data to enrich the patient encounter in the following ways:
Contextualize patient symptoms: When a patient reports new symptoms via the questionnaire, their physician can use the data to determine whether the patient’s symptoms are in line with those experienced by other patients during that point in treatment. The provider can then decide to adjust the patient's care plan or share the aggregated ePRO data view with the patient to reassure them that they are not alone in experiencing those symptoms.
Improve patient education: Patient education is critical to patient experience, yet providers often feel ill-equipped to engage in these conversations. Aggregated ePRO data gives physicians the benchmarks to set patient expectations for treatment and recovery. Before a treatment starts, providers can align a patient with what they should expect 30- or 90-days after treatment based on the outcomes data of other patients.
Train physicians on how to use ePROs for patient care
Physicians aren’t traditionally trained in medical school on how to interpret ePRO data, so MGB offers group trainings to make ePRO data as easy for providers to leverage as possible.
When new questionnaires are launched, providers receive trainings through forums and best practice publications. Key elements of the training include understanding the value of ePROs, the workflows for patients and providers, and how to read and consider both individual and aggregated ePROs in the EMR.
Within the EMR, MGB provides easy-to-understand and easy-to-use visualizations to make the ePRO data meaningful to providers. The intention is that providers shouldn’t have to be data experts to interpret ePROs.
Advice from MGB
When a patient shares an experience, physicians often believe they have the best knowledge to interpret it. We are learning this is misguided thinking. What we see in the clinic is different from what patients are experiencing when they are not with us. PROs help us capture what patients are really experiencing—the real-world data.
Dr. Nadine Jackson McCleary
Mass General Brigham
MGB aims to ensure that the ePRO data influencing care delivery isn’t coming from a homogenous patient sample but a group representative of the diverse patient populations they serve. This required MGB to identify inequities in access that exist in both the technology platform and operations of the program.
Below are three examples of the changes MGB made to ensure their PROMs program was inclusive.
Build for patients with the greatest need
Many digital health tools are designed for patients who have access to health care and few barriers to digital technologies. However, MGB leadership recognized that in order to improve outcomes for all patients, they would need to build for their highest-need patients, especially those without access to specific technologies, and/or who lack digital or health literacy. The ePRO team now consistently reflects on the question of “who is the system built for?” and challenges themselves to build for patients who might be outliers or have the greatest need.
Advice from MGB
If ePRO data is coming from a homogenous population, you can’t use it to make predictions about everyone.
Dr. Nadine Jackson McCleary
Mass General Brigham
Think beyond digital inequities
Often when organizations implement PROMs programs, they focus on addressing digital inequities (e.g., access to smart phones, tablets, internet connectivity). However, it is equally as important to account for the languages and literacy levels of the patients being served. For example, MGB’s patient questionnaires were initially offered only in English and were inaccessible to patients who were visually or cognitively impaired.
In 2020, MGB embarked on a system-wide initiative (United Against Racism) to close disparities and increase health care equity. As part of this effort, MGB’s PROMs program has now translated and built over 500 questionnaires in seven different languages (Spanish, Portuguese, Traditional Chinese, Haitian, Creole, Arabic, and Russian). Additionally, dedicated coordinators are now available to help patients fill out their questionnaires as needed.
Identify and address bias in operations
MGB discovered that bias can exist not just in ePRO platforms and questionnaires but also in survey distribution channels. For example, MGB found that their front desk staff were less likely to offer ePRO tablets to patients of color compared to white patients. Program leaders then met with staff to discuss the problem. By simply pointing out the disparity and providing education, MGB increased tablet distribution from 30% to 86% in two months.
Standing up a PROMs program is an iterative process. To continuously improve the program, PROMs program staff conduct quality assurance checks to ensure program accessibility, usability, and ease. The team tracks a multitude of metrics to evaluate whether patients are appropriately engaging with the program, providers are using the ePRO data to inform the patient encounter, and the organization is using the data to better target quality improvement initiatives.
The quality assurance checks help improve the overall effectiveness of the PROMs program, but they also serve to guide how MGB defines what success looks like—and how the definition of success might evolve. Below is MGB’s current vision for what a successful PROMs program should accomplish.
This article is sponsored by Pfizer. Advisory Board experts wrote the article, conducting the underlying research independently and objectively. Pfizer had the opportunity to review the article.
Create your free account to access 2 resources each month, including the latest research and webinars.
You have 2 free members-only resources remaining this month remaining this month.
Never miss out on the latest innovative health care content tailored to you.