Massachusetts Institute of Technology

Healthcare - Harnessing the World's Collective Intelligence to Cure My Cancer

Thomas W. Malone, Alex (Sandy) Pentland, Drazen Prelec, Peter Szolovits, Josh Tenenbaum, and Marty Tenenbaum[*]

Imagine a complex organism--a worldwide network of many humans and computers--that uses a variety of raw data about individual patients, clinical practices, and basic medical research to make predictions and recommendations for individual patients.   And imagine, further, that these predictions and recommendations get better and better over time as the system is used.   We propose to do some of the essential research needed to help create such a system.  

Consider the following scenario:

John went to his primary care physician, Dr. Smith, with a mole on the sole of his left foot.   The mole had changed shape and grown larger in the past month, and Dr. Smith thought the mole looked suspicious, so she immediately referred John to Dr. Brown.   Brown was a dermatologist participating in an early trial of the Melanoma Network (MNet), a collective intelligence network for melanoma treatment.   A technician in Dr. Brown's office took several high resolution photos of the mole, added these to the detailed electronic medical records Dr. Smith maintained for all her patients, and submitted John's file to MNet.   Within a few minutes, the network returned a result:   Based on the information available so far, the network estimated a 43% probability that the mole was malignant, and recommended a simple biopsy.   Dr. Brown agreed with this recommendation, performed the biopsy in his office, and sent the tissue to be analyzed.

Several days later, the biopsy results were ready, along with the MNet analysis.   Unfortunately, the biopsy was positive; John's mole was, indeed, melanoma.   But the MNet analysis was very encouraging.   MNet evaluated several possible treatments and gave the highest rating to a combination of two treatments:   surgical removal of the mole, and off-label treatment with an obscure drug that most dermatologists had never heard of.   The drug had failed its clinical trials for melanoma because it had only a 2% cure rate.   But the 2% cure rate had occurred in exactly the patients whose melanoma had been on non-pigmented parts of their skin.   Since John's mole was on a non-pigmented part of his skin (the sole of his foot), the estimated success rate of the combined drug and surgical treatment in John's case was 98%.  

What is needed?

Three key elements are needed to make a scenario like this possible:   (1) computer programs to make automated predictions in certain cases, (2) methods for combining predictions from multiple people and computer programs, and (3) an incentive system that motivates people and organizations to contribute their time, information, and other resources to a system like MNet.   Our project will include subprojects in each of these areas.   

Automated predictions.   There is a rich tradition of using mathematical and computational approaches for forecasting future events based on historical and other data:   linear and other forms of statistical extrapolation, multi-variate regression analysis, rule-based systems, neural nets, and other types of data mining and machine learning.   For instance, several of our faculty members have developed new algorithms for making predictions based on patterns in large datasets. [1]  This subproject will focus on developing, evaluating, and then incorporating a number of algorithms that can make automatic predictions based on information available in the system.

Combining predictions .   One promising approach for combining predictions from multiple participants is prediction markets where participants buy and sell predictions about uncertain future events.   The prices that emerge in these markets are often better predictors than opinion polls or individual experts.   Another promising approach (developed by another one of our faculty members) involves a novel statistical technique, called the Bayesian Truth Serum, for aggregating probability assessments from a group of people [2].   For instance, this technique helps identify actual experts, based on their performance.   And it can then be used to scale up the application of their expert diagnostic capabilities to many more cases. We plan to embed these (and other) prediction techniques in a larger framework where participants can see a variety of relevant data before making their predictions about an event, and the current predictions for one event may be inputs to computational or human predictions about other events.

Incentive design.   One promising approach for motivating people to spend time making predictions (or developing programs to make automated predictions) is to use a miniature predicton economy inside a system like MNet.   In one variation, for instance, participants who make correct predictions would be paid for their predictions (in real money or some kind of points), and those who made incorrect predictions would not be paid.   Participants in the final prediction markets could then, in turn, use these payments to pay other participants for information useful in making the final predictions.  

One desirable property of this approach is that people have no incentive to participate in markets when they feel the automated algorithms are doing a good job, but if they feel the algorithms are not taking into account some information the people know, then the people have an incentive intervene.    

This subproject will develop and evaluate a number of alternative ways to motivate people, drawing on rich research traditions in economics, psychology, organizational design, and cognitive science.   For instance, in what situations will non-financial incentives like fun and recognition be more appropriate than financial ones?   And for what kinds of tasks can non-physicians like medical students, undergraduates, and the general public perform well?   

[*]CommerceNet, Palo Alto, CA.

[1] See, for example:   Kemp, C., Shafto, P., Berke, A., and Tenenbaum, J. B. Combining causal and similarity-based reasoning, Advances in Neural Information Processing Systems 19 , in press; and Influence Modeling and Network Discovery, Wen Dong and Alex Pentland, NetSci '07, NYC, May 20-25, also Technical Note 610, http://hd.media.mit.edu.

[2] Drazen Prelec, A Bayesian Truth Serum for Subjective Data, Science, October 15, 2004, pp. 462-466.