Editors' ChoiceNeuroscience

Crowdsourcing Disease Prognosis

See allHide authors and affiliations

Science Translational Medicine  26 Nov 2014:
Vol. 6, Issue 264, pp. 264ec204
DOI: 10.1126/scitranslmed.aaa2069

Amyotrophic lateral sclerosis (ALS), also known as Lou Gehrig’s disease, is a progressive and ultimately fatal neurodegenerative disease affecting 1 in 400 people in which the motor neurons die. There is only one approved treatment but no shortage of clinical trials. One of the challenges in ALS trials is disease heterogeneity. Some patients with ALS progress slowly over many years, whereas others die within a year of diagnosis. This means that evaluating the effects of treatment requires larger sample sizes, more trial centers, trials of longer duration, and greater costs. Moreover, the absence of reliable prognostic tools leaves doctors and patients with the burden of uncertainty.

Küffner et al. now present a new prognostic algorithm to address these shortcomings. Of particular interest is the method used to derive the algorithm, namely crowdsourcing. The authors took advantage of PRO-ACT, an open-access platform containing pooled clinical trial data from over 8600 patients with ALS. Upon offering a $50,000 prize for the most accurate ALS disease progression models, there were 1073 registrants, although only 10 submitted a solution. The authors compared these 10 solutions to one they had created themselves and found that six performed better than their own.

This process yielded many invaluable insights. First, it highlighted the value of bringing fresh perspectives to solve a persistent scientific problem. Second, the authors meticulously described the crowdsourcing process, including registrant surveys identifying motivating and discouraging factors. Third, they compared methodologies across the submitted entries, which included random forest machine learning, Bayesian trees, nonparametric regression, and simple regression. They concluded that tree-based ensemble regression techniques were the most effective. Fourth, the variables represented in the models highlighted known and potentially new relationships to the underlying disease process. Moreover, developing these models was estimated to decrease future clinical trial sizes by approximately 20%, owing to targeted enrollment of a more homogeneous and rapidly progressing patient population. Perhaps most important for the patients was the ability to more accurately offer prognostic information, decreasing some of the uncertainty associated with this disease.

R. Küffner et al., Crowdsourced analysis of clinical trial data to predict amyotrophic lateral sclerosis progression. Nat. Biotechnol. 10.1038/nbt.3051 (2014). [Full Text]

Navigate This Article