Ian Flynn of the University of Pittsburgh and Sean Peters of Middlebury College tested Google’s NotebookLM by converting published research papers into podcast-style audio overviews. In their October paper in the American Geophysical Union journal Perspectives of Earth and Space Scientists, they evaluate both the strengths and the limits of these AI-generated summaries.
The researchers transformed three papers about volcanism on Mars into NotebookLM audio. The sources differed in format: a five-page letter with three figures and one table, a 29-page research paper with nine figures and four tables, and a 23-page review with 11 figures. Google describes the overviews as “deep-dive discussions between AI hosts that provide in-depth summaries of the key topics in your uploaded sources,” and the authors found the results engaging and written in plain language.
Flynn and Peters praised the use of fairly accurate and creative analogies, which can help teaching and make technical work more accessible. Yet every overview contained errors, and mistakes tended to appear toward the end of the audio. Some errors involved clear overreach—for example, one overview extrapolated from a volcanic feature to suggest liquid water and the possibility of life, claims not present in the original paper. The authors see useful roles for these overviews in learning and in demonstrating how to interpret research, but they conclude: “Overall, NotebookLM’s generated audio overviews can be a useful tool for the planetary science community, but it is doubtful to be a replacement for critically reading the source material.”
Difficult words
- overview — short summary of a longer text or workaudio overviews, overviews
- extrapolate — infer beyond given information, often by extending ideasextrapolated
- analogy — comparison that shows similarity between different thingsanalogies
- volcanism — processes related to volcanoes and volcanic activity
- overreach — an excessive or unsupported conclusion or claim
- interpret — explain the meaning of information or results
Tip: hover, focus or tap highlighted words in the article to see quick definitions while you read or listen.
Discussion questions
- How could teachers use AI-generated audio overviews in class while avoiding the risk of errors?
- What steps would you take to check claims made in an AI summary of a research paper?
- Do you think audio summaries make scientific research more accessible to the public? Why or why not?
Related articles
Touchscreens on car dashboards increase driver distraction
A simulator study found that using a car touchscreen while driving makes steering and touchscreen tasks worse. Multitasking reduced lane control and touchscreen accuracy; researchers suggest simple sensors could monitor attention and change the interface.
Zenica School of Comics: Art and Education for Children
The Zenica School of Comics began during the 1992–95 war and has taught around 200 young artists. The school still runs, faces changes from tablets and AI, and the regional comics scene survives through festivals and cooperation.