AI learns why headlines work ^{CEFR B2}

A team at Yale School of Management shows that knowledge-guided generative AI can produce more engaging and trustworthy headlines by learning why those headlines work. Researchers Tong Wang and K. Sudhir, with pre-doctoral research associate Hengguang Zhou, argue that training a model only on winning headlines risks producing clickbait because the model may learn to exploit superficial words rather than underlying reasons for engagement.

Their system has the language model generate competing hypotheses about why one headline outperforms another, then tests those hypotheses across data until a small set of validated explanations remains. The researchers used 23,000 headlines describing 4,500 articles from Upworthy, supplying parts of that dataset and click-through rates to the model. A pre-trained scoring model based on Upworthy’s A/B-test results measured headline quality during evaluation.

After extracting validated hypotheses, the team fine-tuned the LLM so it writes headlines that maximise engagement for the right reasons. They tested the system with about 150 people who judged headlines from three sources: original Upworthy headlines, standard AI-generated headlines, and headlines from the new framework. Human and standard AI headlines were chosen roughly 30% of the time each, while the new model was chosen 44% of the time, and analysis showed the standard AI relied more on sensational language.

The researchers suggest the approach can generate knowledge across fields. Sudhir described work with a company to create personalised AI coaching for customer service agents, where the system could review interactions, propose hypotheses about what works, validate them, and offer advice. They also note inputs need not be only text and could include audio or visual data. The conclusion is that knowledge-guided AI can improve content and make systems more responsible and trustworthy.