TL;DR:
- AI Scientist, developed by Sakana AI and academic labs, can perform the full cycle of research from reading literature to writing and reviewing papers.
- The system is limited to the field of machine learning and lacks the ability to conduct laboratory work.
- Experts praise the openness of the project but note that AI Scientist’s outputs are incremental and it has a popularity bias in referencing papers.
The Dawn of AI in Scientific Research
Imagine a world where science is fully automated. A team of machine-learning researchers has taken a significant step towards this future with the creation of AI Scientist. Developed by Sakana AI in Tokyo, along with academic labs in Canada and the United Kingdom, AI Scientist can perform the entire cycle of research—from reading existing literature to formulating hypotheses, conducting experiments, and even writing and reviewing its own papers.
What is AI Scientist?
AI Scientist is a large language model (LLM) designed to automate the scientific process. It starts by reading the literature on a problem and formulating hypotheses for new developments. It then conducts its own ‘experiments’ by running algorithms and measuring their performance. Finally, it produces a paper and evaluates it through an automated peer review process.
Cong Lu, a machine-learning researcher at the University of British Columbia and co-creator of AI Scientist, explains, “To my knowledge, no one has yet done the total scientific community, all in one system.” The results were recently posted on the arXiv preprint server.
The Potential of AI in Science
Jevin West, a computational social scientist at the University of Washington, praises the project. “It’s impressive that they’ve done this end-to-end,” he says. “And I think we should be playing around with these ideas, because there could be potential for helping science.”
However, the output of AI Scientist is not yet groundbreaking. The system can only do research in the field of machine learning and lacks the ability to conduct laboratory work.
Gerbrand Ceder, a materials scientist at Lawrence Berkeley National Laboratory, notes, “There’s still a lot of work to go from AI that makes a hypothesis to implementing that in a robot scientist.”
How AI Scientist Works
AI Scientist uses a technique called evolutionary computation, inspired by Darwinian evolution. It applies small, random changes to an algorithm and selects the ones that improve efficiency. The system then conducts its own ‘experiments’ by running the algorithms and measuring their performance. After producing a paper, it evaluates it through an automated peer review process. This cycle can then repeat, building on its own results.
Criticisms and Limitations
Some researchers have been critical of AI Scientist’s outputs.
One commenter on Hacker News stated, “As an editor of a journal, I would likely desk-reject them. As a reviewer, I would reject them.”
West also points out that AI Scientist has a reductive view of how researchers learn about their field.
“Science is more than a pile of papers,” he says. “You can have a 5-minute conversation that will be better than a 5-hour study of the literature.”
The Future of Automated Science
Despite its limitations, AI Scientist represents a significant step forward in the automation of scientific research. Tom Hope, a computer scientist at the Allen Institute for AI, notes that current LLMs are not able to formulate novel and useful scientific directions beyond basic combinations of buzzwords. However, he believes that AI could still automate many repetitive aspects of research.
Ceder agrees, stating, “At the low level, you’re trying to analyse what something is, how something responds. That’s not the creative part of science, but it’s 90% of what we do.”
Broadening AI Scientist’s Capabilities
Lu believes that to broaden AI Scientist’s capabilities, it might need to include other techniques beyond language models. Recent results from Google DeepMind have shown the power of combining LLMs with symbolic AI techniques, which build logical rules into a system rather than relying solely on statistical patterns in data.
“We really believe this is the GPT-1 of AI science,” Lu says, referring to an early large language model by OpenAI.
The Debate on AI in Science
The development of AI Scientist feeds into a broader debate about the role of AI in scientific research.
West notes, “All my colleagues in different sciences are trying to figure out, where does AI fit in in what we do? It does force us to think what is science in the twenty-first century — what it could be, what it is, what it is not.”
Comment and Share:
What do you think about the future of AI in scientific research? Could AI Scientist revolutionise the way we conduct research? Share your thoughts and experiences with AI and AGI technologies in the comments below. Don’t forget to subscribe for updates on AI and AGI developments.
You may also like:
- To learn more about AI for science tap here.
