Ian Flynn of the University of Pittsburgh and Sean Peters of Middlebury College tested Google’s NotebookLM by converting published research papers into podcast-style audio overviews. In their October paper in the American Geophysical Union journal Perspectives of Earth and Space Scientists, they evaluate both the strengths and the limits of these AI-generated summaries.
The researchers transformed three papers about volcanism on Mars into NotebookLM audio. The sources differed in format: a five-page letter with three figures and one table, a 29-page research paper with nine figures and four tables, and a 23-page review with 11 figures. Google describes the overviews as “deep-dive discussions between AI hosts that provide in-depth summaries of the key topics in your uploaded sources,” and the authors found the results engaging and written in plain language.
Flynn and Peters praised the use of fairly accurate and creative analogies, which can help teaching and make technical work more accessible. Yet every overview contained errors, and mistakes tended to appear toward the end of the audio. Some errors involved clear overreach—for example, one overview extrapolated from a volcanic feature to suggest liquid water and the possibility of life, claims not present in the original paper. The authors see useful roles for these overviews in learning and in demonstrating how to interpret research, but they conclude: “Overall, NotebookLM’s generated audio overviews can be a useful tool for the planetary science community, but it is doubtful to be a replacement for critically reading the source material.”
Difficult words
- overview — short summary of a longer text or workaudio overviews, overviews
- extrapolate — infer beyond given information, often by extending ideasextrapolated
- analogy — comparison that shows similarity between different thingsanalogies
- volcanism — processes related to volcanoes and volcanic activity
- overreach — an excessive or unsupported conclusion or claim
- interpret — explain the meaning of information or results
Tip: hover, focus or tap highlighted words in the article to see quick definitions while you read or listen.
Discussion questions
- How could teachers use AI-generated audio overviews in class while avoiding the risk of errors?
- What steps would you take to check claims made in an AI summary of a research paper?
- Do you think audio summaries make scientific research more accessible to the public? Why or why not?
Related articles
AI tool helps local autism diagnosis in Missouri
Researchers at the University of Missouri tested the FDA-approved CanvasDx, an AI device, to help primary care evaluate autism where specialty centres are far away. In a study it gave determinate results for 52% of 80 children and matched clinicians' diagnoses.
New ultrasound method improves diagnosis of breast masses
Researchers developed a new ultrasound signal-processing method that distinguishes fluid cysts from solid breast masses. In initial patient tests doctors identified masses correctly far more often than with conventional ultrasound, which could reduce biopsies and follow-ups.
