Researchers compared standard fine-tuning with a different method called Implicit Chain of Thought (ICoT). They looked at how models handle long calculations and whether the models can keep intermediate results during many steps.
Under standard fine-tuning, models with two to 12 layers had less than 1% accuracy on four-digit multiplication. By contrast, the ICoT-trained model reached 100% accuracy. The team found they could read running sums from the ICoT model’s internal states, which shows the model learned to remember useful intermediate values.
The researchers also added a training objective that teaches a model to track running sums. Adding this objective raised a two-layer model’s accuracy to 99% without explicit chain-of-thought supervision.
Difficult words
- fine-tuning — small training to improve a model
- implicit — not directly shown or written
- intermediate — between the first and last steps
- accuracy — how often answers are correct
- internal — inside a system or model
- objective — a goal used during training
Tip: hover, focus or tap highlighted words in the article to see quick definitions while you read or listen.
Discussion questions
- Do you write intermediate results when you do long calculations? Why?
- Would you like a model that remembers intermediate values? Why or why not?
- How can remembering intermediate values help solve a problem?
Related articles
Biodegradable patch may help heart heal after heart attack
Researchers report a biodegradable microneedle patch that delivers interleukin-4 to injured heart tissue. The local treatment encourages healing immune cells, reduces scarring, and may improve heart recovery while avoiding systemic side effects.
Sportellino: a multilingual chatbot for migrants in Italy
A multilingual chatbot called Sportellino launched to help migrants in Italy find public services and practical guidance. It is free, anonymous and available via messaging apps; its information is based on official sources and experts.
High-dose antioxidants may harm sperm and offspring
A study in mice found that regular high doses of antioxidant supplements can damage sperm and change skull and facial development in offspring. Researchers advise caution for men taking large antioxidant doses when planning children.
Egyptian university and pharma join to create Africa’s first biotechnology academy
The American University in Cairo and Minapharm have formed a partnership to set up what the university calls the first African academy for biotechnology. The initiative starts early this year to strengthen education, research and industry links.