MHQP Using Artificial Intelligence to Make Patient Comments More Actionable

(December 2021)

Patient narratives – hearing feedback directly from patients in their own words – is perhaps the most powerful tool for healthcare practices to improve patient experiences. Qualitative feedback can provide personal insights and perspectives that no quantitative data can offer.

Recognizing this fact, MHQP took action a few years ago to begin collecting patient comments as part of our annual statewide Patient Experience Survey. We are now capturing over 35,000 narrative comments in this survey each year.

Now, this year, we have a taken another important step to make this feedback more accessible and digestible to the practices that use these patient stories to direct improvement efforts.

“We used to provide sheets with hundreds of narratives because we had no way to automatically organize this many comments to make the information more easily accessible,” says Amy Stern, MHQP’s Director of Operations and Commercial Survey Programs. “Our participating provider groups had to sift through pages and pages of comments to capture the insights, and it was quite overwhelming.”

Enter “The Machine”

Now, MHQP has started using artificial intelligence (AI) to organize thousands of patient comments. We’ve done this by training what MHQP’s staff members fondly refer to as “The Machine” to think like a human. The Machine is a natural-language processing (NLP) tool that uses machine learning to uncover valuable insights and connections in text.

Raji Rajan, MHQP’s Associate Director of Reporting and Analytics, was tasked with the training.

“Training the machine means we had to manually code comments until the program understood our thinking and the logic behind it,” says Rajan. “It was an iterative process that took several phases until we reached a point where the program can accurately predict what a human would do.”

“This machine learning is a perpetual process,” she says. “The result gets better every time you run it.”

The Result

The result of this endeavor is that MHQP can easily and quickly organize a practice’s comments into the survey’s composite categories (as well as new areas focused on trust, referrals and technology), making the information much more accessible and helpful for improvement efforts.

For example, all of the comments related to the patient’s experience with the office staff are now grouped together in a category called “Office Staff.” All comments about patient-provider communications are now categorized into a “Communications” Category.

“The practice can then quickly identify where they might be having issues,” says Barbra Rabson, MHQP’s president and CEO. “While quantitative scores provide valuable data, the qualitative feedback in the patient’s voice speaks to what is the root cause of what’s behind the data. This gives practices a much deeper understanding of what’s going on and where to focus their energies to improve care. We’re constantly trying to get closer to the patient voice and this is one important step in that direction.”

This step has resulted in national recognition for the effort. Stern and Rajan recently presented on this innovative work at a conference sponsored by the US Agency for Healthcare Research and Quality (AHRQ).

What’s perhaps most exciting is that others can now benefit from MHQP’s AI journey. MHQP would be happy to discuss opportunities to collaborate with organizations or entities seeking to draw actionable insights from patient narrative data. Please contact Amy Stern at astern@mhqp.org if interested.