Never-skilling: Research Reveals AI’s Impact on Junior Developers’ Debugging Skills
A randomised trial found developers who used AI scored nearly two letter grades lower on a quiz about code they had written minutes earlier, with the largest gap observed in debugging.
July 13, 2026 – 12:19 pm
Research published this year has identified a growing concern among employers: never-skilling, where novices never develop proficiency due to limited practice. This is particularly worrying as hiring trends suggest companies are recruiting fewer of these individuals.
The most compelling evidence comes from a randomised controlled trial conducted by Anthropic researchers Judy Hanwen Shen and Alex Tamkin, published in January.
They recruited 52 mostly junior software engineers, split them into two groups, and introduced a Python library (Trio) they were unfamiliar with. One group received an AI assistant while the other coded manually. All participants were then quizzed on the concepts they had used moments before.
The AI group averaged 50%, while the hand-coding group scored 67%. The gap, statistically significant with a p-value of 0.01, represented nearly two letter grades. Although the AI group finished about two minutes faster, this difference was not statistically significant due to some participants spending up to 11 minutes crafting queries.
The most striking finding? The AI group did not debug errors as effectively as their manual counterparts. Debugging, it seems, is a crucial skill that requires hands-on experience.
A Nature Medicine Perspective published in May, led by Duke-NUS Medical School, echoes these concerns from a different perspective. They introduce the term mis-skilling for trainees who uncritically accept AI errors as facts during their clinical training years.
The authors propose a three-phase framework to mitigate this:
- Build competence without AI.
- Teach users to calibrate their scepticism towards AI outputs.
- Introduce the tools under supervision.
The study highlights that how an AI tool is used matters more than its mere presence. In the Anthropic trial, high scorers asked conceptual questions or requested explanations alongside code, while low scorers delegated entirely or let the AI debug for them.
Employers are already factoring this into hiring decisions, as evidenced by Gartner’s findings.