Mark Klein's Home Page

Welcome My Research Curriculum Vita Personal Info Contact Me

The Deliberatorium: Crowd-Based Innovation

The Challenge

Current collective innovation technologies fail badly when dealing with complex problems and large groups. The fundamental challenge, ironically, is super-abundance: crowds generated so many ideas that it becomes difficult to manage the process and harvest the best ideas.

Our Approach

The Deliberatorium system combines ideas from argumentation theory and social computing to help large numbers of people, distributed in space and time, combine their insights to find well-founded solutions for complex multi-stakeholder multi-disciplinary ("wicked") problems.

See this video for a 10 minute introduction to the concepts underlying the Deliberatorium, and follow this link to access the system itself.

Selected Publications

Making Sense of Large-Group Discussion Using Automatically Generated RST-Based Explanations. Social Science Research Network, 2015.

An Embarrassment of Riches. Communications of the ACM, Vol. 57 No. 11, Pages 40-42, 2015.

video presentation (4 minutes) of the European Union-funded CATALYST project, which is applying our work on metrics for guiding open innovation.

video presentation on the Deliberatorium (20 minutes) given at the 2012 Epistemic Democracy Workshop at Yale University

Enabling Deliberations in a Political Party Using Large-Scale Argumentation: A Preliminary Report. Proceedings of the 10th International Conference on the Design of Cooperative Systems.

The Bag of Lemons: High-Speed Idea Filtering for Open Innovation. CCI Working Paper, 2014.

How to Harvest Collective Wisdom on Complex Problems: An Introduction to the MIT Deliberatorium. CCI working paper, 2011.

Enabling Large-Scale Deliberation Using Attention-Mediation Metrics. Journal of Computer-Supported Cooperative Work. 21(4):449-473.

Supporting Collaborative Deliberation Using a Large-Scale Argumentation System: The MIT Collaboratorium. Proceedings of: Directions and Implications of Advanced Computing; Conference on Online Deliberation (DIAC-2008/OD2008). 2008.

Harnessing Collective Intelligence to Address Global Climate Change. Innovations Journal, 2007. 2(3): p. 15-26.

See articles about the Deliberatorium in:


Marco Cioffi, Bain Consulting
Gregorio Convertino, Xerox Research
Cristina Garcia, Fluminense Federal University, Brazil
Ali Gurkan, Ecole Central Paris
Luca Iandoli, University of Naples, Italy
Hajo Reijers, Eindhoven University of Technology, Netherlands
Paolo Spada, Harvard
Catherine Spence, Intel
Michael Winikoff, University of Otago, New Zealand