Human–Machine Hybrid Predictions Led to Better Prognoses of MS, Study Finds

Most patients with multiple sclerosis (MS) experience a relapsing-remitting (RR) phase when the disease starts, followed by a secondary progressive (SP) phase. Researchers in Italy, who recently evaluated the different methods used to predict the duration of the RR phase and the overall disease progression, found that human—machine hybrid predictions led to better prognoses than did machine learning algorithms or groups of humans alone.

A recent study involved a data set of clinical records from 527 visits of 84 outpatients for MS treatment at Sant’Andrea hospital in Rome. Forty-two medical students volunteered to evaluate 50 medical records and estimate the probability that the patient would progress to the SP phase within 180, 360, and 720 days, with scores from 0 (extremely unlikely) to 5 (highly probable). The researchers also calculated predictions using a machine learning algorithm approach, followed by a hybrid prediction that combined human clinical reasoning with the machine learning algorithm approach.

“The palette of disease-modifying treatments is becoming relatively large, in principle opening the possibility to tailor the therapy to meet the specific needs of each patient. Unfortunately, the accuracy of parameters to predict the rate of disease progression remains suboptimal,” the authors explained. “Combination of human and machine predictions into hybrid forecasts exploits human intuitive reasoning and computer classification capabilities, potentially boosting both.”

The results revealed a significant improvement in predictive ability when the predictions were combined together into the hybrid method, rather than relying on the machine learning algorithms or groups of humans alone. In addition, the authors noted that the human predictions alone had lower predictive ability than the machine learning algorithm predictions alone.

“A significant improvement of predictive ability was obtained when predictions were combined in a non-linear manner, with a weight that depends on the consistence of human (or algorithm) forecasts on a given clinical record,” stated the authors. “In spite of the relatively basic machine learning technique used, the small number of students involved and their limited clinical knowledge, this work suggests that hybrid predictions can be useful to improve the prognosis of MS course.”

The researchers emphasized the need for further developments in the ability to combine collective reasoning and machine predictions that would improve the management of MS care. They specifically suggested an additional study that involves more skilled humans, rather than medical students.

“A reliable tool to predict MS progression can be of aid to clinicians to tailor therapy to each patient, but also in clinical trials, to evaluate whether drugs modify the estimated outcome of each enrolled patient, as proposed for ALS [amyotrophic lateral sclerosis],” concluded the study.

Reference

Tacchella A, Romano S, Ferraldeschi M, et al. Collaboration between a human group and articifical intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study [published online December 22, 2017; revised August 1, 2018]. F1000Res. doi: 10.12688/f1000research.13114.2.