Researchers compared standard fine-tuning with a different method called Implicit Chain of Thought (ICoT). They looked at how models handle long calculations and whether the models can keep intermediate results during many steps.
Under standard fine-tuning, models with two to 12 layers had less than 1% accuracy on four-digit multiplication. By contrast, the ICoT-trained model reached 100% accuracy. The team found they could read running sums from the ICoT model’s internal states, which shows the model learned to remember useful intermediate values.
The researchers also added a training objective that teaches a model to track running sums. Adding this objective raised a two-layer model’s accuracy to 99% without explicit chain-of-thought supervision.
Difficult words
- fine-tuning — small training to improve a model
- implicit — not directly shown or written
- intermediate — between the first and last steps
- accuracy — how often answers are correct
- internal — inside a system or model
- objective — a goal used during training
Tip: hover, focus or tap highlighted words in the article to see quick definitions while you read or listen.
Discussion questions
- Do you write intermediate results when you do long calculations? Why?
- Would you like a model that remembers intermediate values? Why or why not?
- How can remembering intermediate values help solve a problem?
Related articles
Traffic and Pollution in Asian Cities
Traffic congestion in many Asian cities raises pollution, lowers living standards and costs economies money. Cities such as Delhi, Bangkok and Metro Manila are expanding metros, switching to electric vehicles and using pricing and technology to reduce traffic.
Depression test works for people with chronic pain
Researchers found that a common eight-question depression test gives similar results for people with and without chronic pain. The study used a large 2019 US health survey and says clinicians can trust positive screens in patients with pain.
AI audio summaries of research can help — and err
Researchers tested Google’s NotebookLM, which turns research papers into podcast-style audio. The summaries were engaging and clearer for teaching, but every audio overview contained mistakes, so the authors advise reading the original papers to check claims.