AI learns why headlines work ^{CEFR B1}

Researchers at Yale School of Management developed a method that teaches a language model why certain headlines perform better. Instead of training only on winning headlines, the model generates competing hypotheses about why one headline is more engaging and then tests those hypotheses on more data. This process aims to prevent the AI from learning shallow cues or clickbait words.

The team used a dataset of 23,000 headlines describing 4,500 articles from Upworthy, a publisher that had run A/B tests on those headlines. They fed parts of that dataset to the model together with click-through rates and used a pre-trained scoring model based on Upworthy’s A/B-test results to measure headline quality during evaluation.

After validating explanations, the researchers fine-tuned the model so it would maximise engagement for the right reasons. In human evaluations, the new system’s headlines were chosen more often than standard AI headlines. The researchers also describe possible applications such as personalised AI coaching for customer service, and they note the inputs could include audio or video as well as text.