Researchers at Yale School of Management developed a method that teaches a language model why certain headlines perform better. Instead of training only on winning headlines, the model generates competing hypotheses about why one headline is more engaging and then tests those hypotheses on more data. This process aims to prevent the AI from learning shallow cues or clickbait words.
The team used a dataset of 23,000 headlines describing 4,500 articles from Upworthy, a publisher that had run A/B tests on those headlines. They fed parts of that dataset to the model together with click-through rates and used a pre-trained scoring model based on Upworthy’s A/B-test results to measure headline quality during evaluation.
After validating explanations, the researchers fine-tuned the model so it would maximise engagement for the right reasons. In human evaluations, the new system’s headlines were chosen more often than standard AI headlines. The researchers also describe possible applications such as personalised AI coaching for customer service, and they note the inputs could include audio or video as well as text.
Difficult words
- hypothesis — An idea that can be tested with datahypotheses
- validate — Check that a result or explanation is correctvalidating
- fine-tune — Adjust a model to improve its performancefine-tuned
- click-through rate — Proportion of users who click a linkclick-through rates
- engagement — Level of user interest or interaction with content
- A/B test — Compare two versions to see which works betterA/B tests, A/B-test
Tip: hover, focus or tap highlighted words in the article to see quick definitions while you read or listen.
Discussion questions
- Do you think a system that generates and tests hypotheses will make better headlines? Why or why not?
- How could personalised AI coaching help customer service agents in your view?
- How might including audio or video inputs change the headline suggestions the model makes?
Related articles
Moharaj Sharma: poet, journalist and documentary maker
Moharaj Sharma is a Nepali poet, journalist and documentary maker. He works at AP1 Television, studies Nepali language roots, and has made a documentary about Nepali-speaking Bhutanese refugees while writing poetry about identity and social change.
January 2025 Los Angeles wildfires and the rise in virtual health visits
A study of 3.7 million Kaiser Permanente members found that the January 2025 Los Angeles wildfires caused large increases in virtual care, especially for respiratory and cardiovascular symptoms, and raised other outpatient visits.