QualPro Effects Process Improvement with Multivariable Testing & Big Data
Results are “preliminary but promising”with the use of some-
thing called “multivariable testing” (MVT) to identify and
implement ways to improve patient satisfaction at Blount Memorial
Hospital, says Richard Hall, RN, MBA, chief nurse executive and
assistant administrator.
Using MVT, Hill explains, the 304-bed hospital is simultaneously
testing a number of improvement ideas, individually and in combi-
nation, to determine which activities result in improvement, which
don’t have any effect, and which actually have a negative impact.
Blount Memorial in Maryville, TN, is working with QualPro, the
Knoxville, TN-based company that developed the MVT process, on
a six-month project focusing on its emergency department (ED) and
operating rooms.
One test involves next-day follow-up calls to patients who have
been treated in the ED, Hall says. “We do call-backs anyway to talk
about whether they have any questions or if Johnny still has a fever.”
Now the staff member making those calls also asks about pain
management and about the customer’s overall satisfaction with the
ED experience, as well as whether he or she would recommend the
facility, he adds.
Compared to traditional ways of measuring customer satisfaction,
Hall says, “we have a month’s worth [of data] in one day. We see 120
to 140 people a day in the ED, so that makes it statistically significant
for us.”
As expected, he notes, two things the hospital already was doing
were validated as improving throughput and patient satisfaction.
“One was having the ED physicians write admission orders, based
on signs and symptoms, rather than waiting for the specialist, and the
other is that we have an admissions holding area for patients we’re
not sure are going to be admitted.”
In another instance, the variable being tested was successful in two
ways, one of which was totally unexpected, Hall says.
The hospital developed scripts for ED personnel to use in explain-
ing to patients the reasons for their wait times, and to lay out expecta-
tions for what the ED visit would entail, such as “seeing the physi-
cian, asking questions, having tests run,” he notes.
The idea was to enlighten people as to why they were waiting,
but in the process of doing that, staff became more focused on the
need to keep people from having to wait, Hall says. As a result, he
adds, “they got more creative in getting people back to the treatment
room,” which resulted in improved throughput times.
“Without this system,” he adds, “we wouldn’t have discovered
that. You make changes all the time, but you don’t always know if
they will result in improvement.”
Historically, process improvement efforts involved “having a
consultant come in, say, ‘this is the problem,’ and then, ‘Here is what
has worked in other hospitals—we think it will work here,’” recalls
'Multivariable testing' aids process improvement
Statistical analysis is key
Samuel Evans, MD, the hospital’s medical director and administra-
tor of the OR piece of the Blount Memorial project.
“You end up looking at ideas, saying, ‘Try this,’ and if that doesn’t
work, ‘Try something else,’ and eventually you get frustrated and
quit.”
By testing several ideas at the same time, he says, “you have proof
of what is helping and what’s not. The technique used is very inten-
sive on the statistical side.”
QualPro uses an MVT categorization process, a company news re-
lease states, “to narrow test ideas to only those that are practical (easy
and safe to test and implement with current resources), fast (quick
to implement with current resources), and cost-free (no increase in
operating cost or capital).”
Experience has shown, Evans notes, “that about 23% [of test ideas]
will help, about the same percentage will hurt, and the rest of the
variable will do nothing. There is no way to predict which will fall
into which category.”
To prepare for the OR portion of the project, he says, sessions
were held with surgeons, anesthesiologists, and OR staff. “We asked,
‘What are your problems and what do you think will help?’We took
all the ideas everybody had and analyzed them and came up with 23
different variables to test.
“One day we test two or three variables, the next day we change
and do two more, an we do that for 30 days,” Evans continues. “Then
we collect all the data and do a statistical analysis and say, ‘It appears
this variable is helping, this one is also helping, this is not helping at
all and is actually hurting.’”
At that point, some of the variables are eliminated, and there is
another series of tests on the smaller group, he adds, “and we see
what helps.”
One of the tests involved notifying surgeons the day before to
verify scheduled procedures, Evans says. As a result of this advance
check, he notes, “sometimes [the surgeon] decides to change the
plan, or discovers there is an error in how it was recorded.”
Catching such glitches a day in advance, he notes, prevents last-
minute cancellations that irritate patients and family members, delay
turnaround time, and cost money “because the room is set up for one
patient and you’re changing to another.”
“One of the thing we’re doing to improve customer satisfaction is
to be sure to communicate with family members so they have a bet-
ter indication of when to be here, what time the surgery will begin,
and when it will be over,” Evans says. “We look for ways to stimu-
late staff so they know to communicate that information at certain
points.”
Hall points out that “while the literature is full of good ideas” for
process improvement, the difficulty lies in identifying those that
can be implemented and will actually result in improvement for the
patient.
“We’ve got to make our resources count to the fullest for our
patients and maximize all opportunities to do so,” he adds. “This
system helps us know in advance whether long-term implementation
of those resources will make sense.”
HOSPITAL ACCESS MANAGEMENT™ / July 2006