Generative AI and Collective Intelligence
Several CCI research projects examine how generative AI systems can enhance the collective intelligence of human-computer groups.
Supermind Ideator
Previous efforts to support creative problem-solving have included techniques (such as brainstorming and design thinking) to stimulate creative ideas, and software tools to record and share these ideas. Now, generative AI technologies can suggest new ideas that might never have occurred to the users, and users can then select from these ideas or use them to stimulate even more ideas.
Supermind Ideator was created to supercharge creative ideation by harnessing these new generative capabilities. The system uses a large language model and adds prompting, fine tuning, and a user interface specifically designed to help people use creative problem-solving techniques. Some of these techniques can be applied to any problem; others are specifically intended to help generate innovative ideas about how to design groups of people and/or computers (“superminds”).
We have conducted preliminary evaluation to understand early experiences of people using this system and are continuing more thorough empirical evaluations to understand how Ideator augments individuals and teams working on creative problem solving and ideation. We have also created a waitlist where we have been slowly opening up the application to eager beta-testers who have now created more than 10,000 ideas using the system.
Researchers
Steven Rick, Gianni Giacomelli, Haoran Wen, Robert Laubacher, Nancy Taubenslag, Jennifer Heyman, Max Sina Knicker, Younes Jeddi, Hendrik Maier, Stephen Dwyer, Pranav Ragupathy, Thomas Malone
Publications
Jennifer L Heyman, Steven R Rick, Gianni Giacomelli, Haoran Wen, Robert J. Laubacher, Nancy Taubenslag, Pranav Ragupathy, Jared Curhan, Thomas W. Malone, Max Knicker, and Younes Jeddi. Supermind Ideator: How Scaffolding Human-AI Collaboration Can Increase Creativity. Collective Intelligence, December 2024.
Jennifer L Heyman, Steven R Rick, Gianni Giacomelli, Haoran Wen, Robert Laubacher, Nancy Taubenslag, Max Knicker, Younes Jeddi, Pranav Ragupathy, Jared Curhan, and Thomas Malone. Supermind Ideator: How Scaffolding Human-AI Collaboration Can Increase Creativity. CI ’24: Proceedings of the ACM Collective Intelligence Conference, June 2024.
Steven R. Rick, Gianni Giacomelli, Haoran Wen, Robert J. Laubacher, Nancy Taubenslag, Jennifer L. Heyman, Max Sina Knicker, Younes Jeddi, Hendrik Maier, Stephen Dwyer, Pranav Ragupathy, Thomas W. Malone. Supermind Ideator: Exploring generative AI to support creative problem-solving. ArXiv. Nov 2023.
Turing test for human-computer systems
The Turing test for comparing computer performance to that of humans is well known, but, surprisingly, there is no widely used test for comparing how much better human-computer systems perform relative to humans alone, computers alone, or other baselines. Here, we show how to perform such a test using the ratio of means as a measure of effect size. We demonstrate the use of this test in three ways.
First, in an analysis of 79 recently published experimental results, we find that, surprisingly, over half of the studies find a decrease in performance, the mean and median ratios of performance improvement are both approximately 1 (corresponding to no improvement at all), and the maximum ratio is 1.36 (a 36% improvement).
Second, we experimentally investigate whether a higher performance improvement ratio is obtained when 100 human programmers generate software using GPT-3, a massive, state-of-the-art AI system. In this case, we find a speed improvement ratio of 1.27 (a 27% improvement).
Third, we find that 50 human non-programmers using GPT-3 can perform the task about as well as—and less expensively than—the human programmers. In this case, neither the non-programmers nor the computer would have been able to perform the task alone, so this is an example of a very strong form of human-computer synergy.
Our work in this project continues to synthesize human-AI experiments, search for patterns of synergy in human-AI groups, and conduct novel studies using generative AI to augment human abilities.
People
Michelle Vaccaro, Andres Campero, Jaeyoon Song, Haoran Wen, Abdullah Almaatouq, Thomas W Malone
Publications
Michelle Vaccaro, Abdullah Almaatouq and Thomas Malone, When combinations of humans and AI are useful: A systematic review and meta-analysis, Nature Human Behaviour, October 2024.
Andres Campero, Michelle Vaccaro, Jaeyoon Song, Haoran Wen, Abdullah Almaatouq, Thomas W. Malone, A test for evaluating performance in human-computer systems, ArXiv, June 2022.
Videos
What is an Analogue of the Turing Test for Human-Computer Systems?
DesignAID
Designers often struggle to sufficiently explore large design spaces, which can lead to design fixation and suboptimal outcomes. We designed DesignAID, a generative AI tool that supports broader design space exploration, in order to help address this problem.
By using large language models to produce a range of diverse ideas expressed in words, and then using image generation software to create images from these words, we create human-computer teams that can rapidly create a diverse set of visual concepts without time-consuming drawing.
In a study with 87 crowd-sourced designers, we found that designers rated the automatic generation of images from words as significantly more inspirational, enjoyable, and useful than a conventional baseline condition of image search using Pinterest.
Surprisingly, however, we found that automatically generating highly diverse ideas had less value. For image generation, the high diversity condition was somewhat better in inspiration but no better in the other dimensions, and for image search it was significantly worse in all dimensions.
Researchers
Alice Cai, Steven Rick, Jennifer Heyman, Yanxia Zhang, Alexandre Filipowicz, Matthew Hong, Matthew Klenk, Thomas Malone
Publications
Alice Cai, Steven R Rick, Jennifer L Heyman, Yanxia Zhang, Alexandre Filipowicz, Matthew Hong, Matt Klenk, Thomas W. Malone. DesignAID: Using Generative AI and Semantic Diversity for Design Inspiration, ACM Collective Intelligence Conference 2023, https://dl.acm.org/doi/10.1145/3582269.3615596
Links
Presentation at 2023 ACM Collective Intelligence Conference
Press
MIT Sloan School of Management, A generative AI tool to inspire creative workers, February 14, 2024
Using algorithms and generative AI to create better matches between employers and workers
A project sponsored by the MIT and Accenture Convergence Initiative for Industry and Technology sought to develop an innovative labor market recommender system that could better match employers and workers. The project had two workstreams:
- Building a software algorithm to match employers and job candidates and also to recommend actions both parties could take to encourage more positive employment outcomes.
- Design of organizational structures/processes that elicit information needed by the algorithm, incentivize key stakeholders to contribute that information, and use the information effectively once it has been gathered.
The algorithm stream built on prior MIT research on how information systems can encourage more efficient and equitable labor market outcomes.
The organizational workstream used the Supermind Design framework, a structured method for inventing more intelligent organizations, and Supermind Ideator, a generative-AI based software platform, that seeks to spur creative solutions to organizational challenges. Ideator relies on custom-engineered prompts, informed by Supermind Design principles, to Open AI’s large language models (LLMs).
CCI researchers
Thomas W. Malone, John J. Horton, Robert Laubacher, Steven Rick, Jennifer Heyman, Younes Jeddi
Accenture researchers
Nitu Nivedita, Thomas Hancock, Terry Huang
Publications
Click on thumbnail below to view the CCI-Accenture white paper.
Click on thumbnail to view the CCI-Accenture white paper.
Robert Laubacher, Jennifer Heyman, Steven R. Rick, Younes Jeddi, Thomas Hancock, Terry Huang, Nitu Nivedita, and Thomas W. Malone. Using Supermind Design and Generative AI to reinvent the Future of Work. MIT and Accenture Convergence Initiative White Paper, July 2024.