Researchers compared standard fine-tuning with a different method called Implicit Chain of Thought (ICoT). They looked at how models handle long calculations and whether the models can keep intermediate results during many steps.
Under standard fine-tuning, models with two to 12 layers had less than 1% accuracy on four-digit multiplication. By contrast, the ICoT-trained model reached 100% accuracy. The team found they could read running sums from the ICoT model’s internal states, which shows the model learned to remember useful intermediate values.
The researchers also added a training objective that teaches a model to track running sums. Adding this objective raised a two-layer model’s accuracy to 99% without explicit chain-of-thought supervision.
Difficult words
- fine-tuning — small training to improve a model
- implicit — not directly shown or written
- intermediate — between the first and last steps
- accuracy — how often answers are correct
- internal — inside a system or model
- objective — a goal used during training
Tip: hover, focus or tap highlighted words in the article to see quick definitions while you read or listen.
Discussion questions
- Do you write intermediate results when you do long calculations? Why?
- Would you like a model that remembers intermediate values? Why or why not?
- How can remembering intermediate values help solve a problem?
Related articles
Dementia rising in Africa as researchers seek answers
Dementia is increasing in Africa as populations age. Research and evidence in the region are limited, so scientists study genetics, new detection tools and community measures while working with traditional healers to reduce stigma.
Egyptian university and pharma join to create Africa’s first biotechnology academy
The American University in Cairo and Minapharm have formed a partnership to set up what the university calls the first African academy for biotechnology. The initiative starts early this year to strengthen education, research and industry links.
