Emerson Professors Want to Revolutionize Research Statistics
For roughly a century, clinicians and researchers have relied on “significance testing”—a statistical method of deductive reasoning—to make diagnoses and test treatments. There’s just one problem with that, according to Emerson College professors Eiki Satake and David Maxwell.
Significance testing doesn’t tell you what you want to know.
Satake, professor of mathematics and statistics in the Institute for Liberal Arts and Interdisciplinary Studies, and Maxwell, professor emeritus in the Department of Communication Sciences and Disorders (CSD), want to upend the way research methods are taught in clinical classrooms by introducing students to Bayesian statistics, a system of inductive reasoning that they say is much better suited to clinical uses.
“Traditional statistics involve…looking for the chance of a random error occurring,” said Maxwell. “What you actually want to know is a probability or likelihood for some kind of an outcome, like the outcome of a particular therapy, A, being better than the outcome of therapy B.”
Maxwell and Satake, both of whom have written about so-called evidence-based practice in numerous papers and books, are talking about Bayesian statistics at the annual American Speech-Language-Hearing Association Convention in Philadelphia this week, where they will try to convince their peers to revise statistical research classroom requirements in the field of communication disorders.
They expect pushback, partly because, as Satake said, “people by nature are very conservative,” but also because Bayesian statistics is much harder to teach than traditional statistics.
In significance testing, which is deductive, a clinician will start with a hypothesis (“Therapy A will be most effective.”), then reframe it in the negative (“Therapy A will not work.”). She will then examine a patient’s symptoms and ask herself, if my hypothesis is false, what is the probability that these symptoms would be present?
In evidence-based practice, which is inductive, a clinician looks at a patient’s symptoms and, based on her own experience, forms a hypothesis as to what the best treatment will be. If that sounds ridiculously simple, it’s probably because that’s the way the human mind works, Satake said. Bayesian statistics—named for the Rev. Thomas Bayes, the 18th-century English clergyman who developed the theorem—is just a way to quantify that mathematically.
“We are naturally Bayesian, instinctively, but in the classroom they never formulate that process,” he said.
As a clinician becomes more experienced, she will have more data backing her hypothesis, which she can then use to fine-tune her conclusions. Bayesian statistics allows for new information to enter into the equation cumulatively.
“In traditional research, you get a yes or no answer. It was effective or it wasn’t,” Maxwell said. “[Bayesian statistics] allows me to update my belief and to use it…. The precision of it is much greater than what we’re able to gain through traditional statistics.”
But that introduction of subjectivity into the model (a clinician’s personal experience), while ultimately a strength from a diagnostic standpoint, makes it harder to quantify scientifically. Unlike traditional statistics, Bayesian statistics requires a significant amount of calculus to master, and that tends to scare off a lot of would-be instructors and students.
Satake is currently developing a computer program that would take some of the sting out of teaching Bayesian in standard CSD graduate courses, and he and Maxwell are hoping the method will become more widely taught across disciplines.
Maxwell said that early in the 20th century, objectivity in the form of significance testing became the sine qua non in the scientific and medical worlds, but we’re starting to see some long-accepted “objective” studies questioned, most recently the heart benefits of HDL or “good” cholesterol.
“We’re beginning to see that studies in medicine, like maybe a third to half of them, are producing inaccurate results,” Maxwell said. “That’s what’s come from this very flawed model.”
Satake and Maxwell’s focus is on communication sciences and disorders, but Bayesian statistics has applications in a wide range of fields. An extremely popular mutual fund uses the model to build its portfolio, Satake said. And (now you tell us) it can be useful in political and public opinion polling.
Satake said if Emerson’s Communication Sciences and Disorders Department eventually is able to incorporate Bayesian into its curriculum, it might have implications, not just for clinical work, but also for admissions.
“If we [can say] Emerson has a very innovative research methodology that Syracuse University doesn’t have, New York University doesn’t have, the University of Pennsylvania doesn’t have, that will be a very appealing point, and that may increase the number of graduate applicants,” he said.
Categories
