Mark Klein's Home Page

Welcome My Research Curriculum Vita Personal Info Contact Me

The Deliberatorium: Crowd-Based Innovation

The Challenge

Current collective innovation technologies fail badly when dealing with complex problems and large groups. The fundamental challenge, ironically, is super-abundance: crowds generated so many ideas that it becomes difficult to manage the process and harvest the best ideas.

Our Approach

The Deliberatorium system combines ideas from argumentation theory and social computing to help large numbers of people, distributed in space and time, combine their insights to find well-founded solutions for complex multi-stakeholder multi-disciplinary ("wicked") problems.

A 10 minute introduction to the concepts underlying the Deliberatorium

Go to to access the system itself.

Selected Publications

A Review of Crowd-Scale Online Deliberation Technologies, MIT Working Paper, August 2015.

The Bag of Lemons: High-Speed Idea Filtering for Open Innovation. Decision Support Systems Journal, in press.

Making Sense of Large-Group Discussion Using Automatically Generated RST-Based Explanations. MIT Working Paper, June 2015

A Roadmap for Open Innovation Systems. Journal of Social Media, 1(2), 2015.

An Embarrassment of Riches. Communications of the ACM, Vol. 57 No. 11, Pages 40-42, 2015.

video presentation on the Deliberatorium (20 minutes) given at the 2012 Epistemic Democracy Workshop at Yale University

Enabling Deliberations in a Political Party Using Large-Scale Argumentation: A Preliminary Report. Proceedings of the 10th International Conference on the Design of Cooperative Systems. May 2012.

How to Harvest Collective Wisdom on Complex Problems: An Introduction to the MIT Deliberatorium. CCI working paper, 2011.

Enabling Large-Scale Deliberation Using Attention-Mediation Metrics. Journal of Computer-Supported Cooperative Work. 21(4):449-473.

Popular Press Articles


Jared Bataillon
Abraham Bernstein, University of Zurich
Nancy Bordier
Marco Cioffi, Bain Consulting
Jeffrey Conklin
Gregorio Convertino, Xerox Research
Anna de Liddo, Open University
Cristina Garcia, Fluminense Federal University, Brazil
Ali Gurkan, Ecole Central Paris
Luca Iandoli, University of Naples, Italy
Yiftach Nagar
Ivana Quinto
Hajo Reijers, Eindhoven University of Technology, Netherlands
Carlo Savoretti
Paolo Spada, Harvard
Simon Buckingham Shum
Catherine Spence, Intel
Mark Tovey
Cyril Velikanov
Michael Winikoff, University of Otago, New Zealand