Over-Reliance on AI in Math Homework: The Evidence

Key Points

• In a randomised trial of nearly 1,000 high school students published in PNAS, those given plain ChatGPT for math practice scored 17% lower than classmates who never had it once the tool was taken away.

• Pew Research Center found that 43% of US teens aged 13 to 17 have used an AI chatbot to help solve a math problem for school, making math the second most common homework use after researching a topic.

• Across 13 frontier models, the median gap between performance on hard solved math problems and genuinely unsolved ones is 82.9 percentage points, so benchmark scores measure familiarity rather than reasoning.

• Analysis of a decade of math platform logs found study time on AI-solvable problems fell 31% for high school students but showed no detectable change for Grade 5, meaning the risk tracks unsupervised access rather than age.

• No study has measured whether school-age children can detect incorrect AI-generated math, so the central question parents ask has no research answer yet.

Students given plain ChatGPT scored 48% higher on practice and 17% lower on the unaided exam. Preregistered randomised trial, nearly 1,000 students, grades 9 to 11. Bastani et al., PNAS, June 2025.

Start with the claim this article is not going to make.

Math scores are falling across the rich world. The OECD average fell to 463 in PISA 2025, the lowest in the programme's history, down from 485 in 2015. It is tempting to draw a line from that to the chatbot in every teenager's pocket.

The timeline does not allow it. The single largest drop in PISA's history, 15 points, was measured in the 2022 assessment, which was administered during 2022. ChatGPT was released on 30 November 2022.

The OECD notes that in two decades its math average had never moved more than four points between consecutive cycles. That extraordinary fall happened before generative AI reached students, and the cycle since has fallen by less.

So AI did not cause the math decline. The honest question is narrower and more useful: when a child uses AI on their math homework, what happens to what they learn?

That question does have answers, and some of them are unusually good.

How we built this: we opened and verified more than 40 primary sources, meaning peer-reviewed papers, official statistics, regulator documents and raw benchmark data files. We downloaded and analysed the benchmark files ourselves. Where an effect size could not be read at source, we leave it out rather than repeat it second-hand.

How Many Children Use AI for Math Homework?

43% of US teens aged 13 to 17 have used an AI chatbot to help solve a math problem for school. Among teens who use chatbots for schoolwork at all, 79% have used one on math, which makes it the second most common use after researching a topic.

That figure comes from Pew Research Center fieldwork in autumn 2025, published in February 2026. As far as we can establish, it is the only credible national measure anywhere of children using generative AI on a specific school subject.

It also needs its corrective, because it is an "ever used" figure. Asked how much of their schoolwork they do with AI help, 42% of those teens said "a little" and 39% said "some". Only 5% said all or almost all. Across all US teens, 10% do all or most of their schoolwork with AI, and 45% have never used it that way.

The sharper measure is dependency. Common Sense Media found that 16% of children who use AI for homework help report difficulty starting or finishing homework when they cannot use it, rising to 28% among daily users. A separate finding in the same research: 25% of teens who get an answer from AI use it as-is, unchanged.

One trend you will see elsewhere is wrong. Pew's ChatGPT-for-schoolwork figure went from 13% of teens in 2023 to 26% in 2024, and press coverage often extends that line to 54% in 2025. The 2025 question is different, covering any AI chatbot rather than ChatGPT specifically. Those are not comparable waves, and Pew does not present them as one.

What Happens When the AI Is Taken Away

The best study on this question is a preregistered randomised controlled trial published in PNAS in June 2025, run across roughly 50 classes in a large Turkish high school with nearly 1,000 students in grades 9 to 11.

Students were randomised by classroom into three groups: plain GPT-4 in a chat window, a guardrailed tutor whose prompt contained teacher-written solutions and told it to give hints without answers, and a control group with textbooks and no devices. Every session had an identical teacher lecture, then randomised practice, then a closed-book paper exam with no devices, on problems deliberately matched to the practice problems.

The plain ChatGPT group did substantially better while holding the tool and significantly worse than the control group once it was gone. The guardrailed tutor did far better on practice and left no measurable harm.

Note what the second row does not say. The tutor group did not learn more. The authors state it directly: "we still do not observe a positive effect."

Guardrails stopped the damage. They did not produce learning. The paper's actual title, often shortened in coverage, is "Generative AI without guardrails can harm learning".

A second study, published at COLM 2026 by researchers at Carnegie Mellon, Berkeley, Oxford, MIT and UCLA with no industry funding, ran three randomised trials with 1,222 participants. Two used fractions. The AI was removed and participants were tested unaided.

The first fractions experiment found a solve rate of 0.57 with prior AI access against 0.74 without, an effect size of −0.45. The researchers then ran a pre-planned replication that fixed a confound in their own design. The effect survived but roughly halved, to −0.20. Their separate measure of persistence failed to replicate at all.

That second paragraph is why the study is believable. A team reporting that their own effect shrank by half, and that one of their measures did not hold up, is reporting a finding rather than defending a headline. One caveat travels with it: those participants were paid adults, not children.

Why It Happens: Children Copy, They Do Not Read

The obvious explanation is that the AI gives wrong answers and students absorb them. The PNAS researchers tested that explanation and it failed.

They queried the model ten times on each of the 57 practice problems using the most common student message, "What is the answer?". It gave a correct answer only 51% of the time, with logical errors in 42% of cases and arithmetic errors in 8%.

But those errors did not behave the way the misinformation theory predicts. The model's logical error rate strongly hurt practice scores and then did not carry over to the matched exam problems at all. Arithmetic errors hurt practice about as much as logical ones, which should not happen if students were reading the working, because teenagers can catch a slip in arithmetic.

The authors' conclusion: "Both these results suggest that students are simply copying answers from GPT Base."

The interaction logs say the same thing. In the first session, 67% of messages to the plain model either repeated the question text or asked for the answer. In the guardrailed group that figure fell to 37%.

The most uncomfortable finding is about awareness. Students who used plain ChatGPT performed worse on the exam and did not perceive that they had performed worse or learned less. The guardrailed group believed they had done significantly better when they had not. Whatever signal tells a child that they have understood something, holding a chatbot detaches it from whether they actually have.

There is a laboratory result that explains the mechanism. A randomised trial in Computers in Human Behavior measured three kinds of mental effort and found that germane cognitive load, the effortful kind that constitutes learning, fell furthest when students used an LLM. That drop fully mediated the loss in reasoning quality. The students felt the work get easier, and the part that got easier was the learning.

Can the AI Even Do the Math?

Benchmark scores suggest it can. We went to the underlying data and found that those scores are close to meaningless for this question.

Epoch AI publishes a public dataset of model evaluations. We downloaded it and compared every model evaluated on both FrontierMath Tier 4, which holds very hard problems that have known solutions, and FrontierMath Erdős, which holds 68 problems mathematics has never solved.

All 13 models we checked score lower on unsolved problems than on the hardest solved tier. The median gap is 82.9 percentage points, 10 of the 13 have a gap above 70 points, and 8 of the 13 score exactly zero on Erdős. Several of these evaluations ran within days of each other, so this is not a version artefact.

A benchmark score measures familiarity with problem types, not mathematical reasoning. A child's homework sits firmly on the familiar side, which is exactly why the tools usually appear to work.

Three further findings matter more for homework than any benchmark.

Changing only the numbers degrades performance. Research presented at ICLR 2025 found accuracy declines when a word problem's numerical values alone are altered. A child's homework is, by construction, a fresh instantiation with different numbers.

A single irrelevant sentence can collapse accuracy. The same researchers added one plausible but inconsequential clause to problems and recorded drops of up to 65 percentage points. Supplying eight worked examples of the same question, making the irrelevance obvious, left performance within the margin of error. Real homework questions are full of narrative detail that never enters the calculation.

And the failures are invisible. In a study published in PLOS ONE, 32% of ChatGPT-generated math hints failed quality screening, rising to 47% for intermediate algebra. The authors' description of the failures is the single most useful sentence in this literature: "All disqualified hints contained incorrect answers and incorrect solution steps. None had inappropriate language, poor spelling, or grammatical errors."

Nothing about a wrong answer looks wrong. The only thing wrong with it is the math.

The model cannot reliably mark the work either. Research presented at EMNLP 2025 asked models to find the first error in a real student's step-by-step solution. A model scoring 96.8% on grade-school math found the child's first wrong step only 49.9% of the time, and about 65% even when handed the correct answer. The authors showed statistically that solving ability and error-spotting ability are only weakly related.

Can Children Spot the Errors? Nobody Has Measured It

This is the question parents and teachers actually ask, and it has no answer.

We searched specifically for it. No study measures the rate at which school-age children detect incorrect AI-generated mathematics. The quantified detection rates that exist come from university undergraduates and adult professionals.

The closest figure comes from a 2026 study of undergraduates in a quantitative reasoning course, ages 19 to 54. Asked to judge whether ChatGPT solutions were correct, they detected errors at 3.61 out of 10 at baseline, roughly 36%.

That same study carries the most hopeful finding in this entire article. After four weeks of structured error analysis, where students worked in groups judging, classifying and correcting AI output, detection rose to 7.72 out of 10, roughly 77%, with a large effect size. The study had no comparison group and a single scorer, so treat the size with caution. The direction is still worth noticing: spotting machine errors is a teachable skill, and teaching it took four weeks.

What we do know about children is narrower and concerns whether they check at all. A study of 98 Grade 9 students using an AI tutor during a curriculum math activity coded 808 of their chat turns. Turns in which students evaluated the AI's response had a median of 0%. Requests to verify their own understanding also had a median of 0%, despite 69.7% having said beforehand that they wanted the AI to check their understanding.

Stated intentions did not predict behaviour. That study is a preliminary preprint whose coding was done by a language model with human validation still in progress, so it is suggestive rather than settled.

The nearest evidence about children and machine overtrust is a 2018 study in Science Robotics in which children aged 7 to 9 conformed to a group of robots giving obviously wrong answers on a visual task, while adults resisted. It involves robots and line lengths, not chatbots and fractions. It is a proxy, and should be read as one.

So the honest position: we know children rarely check, we know adults detect roughly a third of errors unaided, and we know the skill can be taught. We do not know how often a twelve-year-old catches a confidently wrong quadratic.

The Age Gradient: Supervision Matters More Than Age

The most practically useful finding comes from a decade of logs on a school math platform, covering about 3.2 million learning interactions from Grade 5 to College Algebra.

The researchers split problems into those that can be typed into a chatbot and those that require manipulating graphs on screen, then measured study time before and after ChatGPT's release.

Time spent on graph-based problems that a chatbot cannot easily take stayed flat in every age group across the whole period. That internal control is what makes the pattern persuasive, because a general decline in attention or curriculum change would have moved both.

A second test is stronger still. In non-proctored assessments, response times on AI-solvable items fell by about 12%. In proctored assessments, the effect was statistically indistinguishable from zero.

Read those two findings together and the conclusion is not about age. It is about supervision and unsupervised device access. The effect appears where children work alone and disappears where someone is watching. A ten-year-old doing homework at the kitchen table behaves differently from a sixteen-year-old with a phone in their bedroom, and the data shows it.

This study needs its labels. It is a preprint that has not been peer reviewed, four of its five authors are employees of the company that owns the platform and the fifth is a former intern there, and its retention finding is marginal, clearing significance at p=0.037 on 6,698 observations. We report the behavioural finding, which rests on the full dataset and two falsification tests, and we do not lean on the retention claim.

What Schools Actually Allow

Here is the structural gap, and it is the finding most likely to be useful to anyone writing policy.

In England, nothing in the Department for Education's guidance on generative AI is enforceable against a pupil. The guidance says schools "are free to make their own choices", and on this specific question it says only that schools "may wish to review homework policies, and other types of unsupervised study to account for the availability of generative AI."

The enforceable rule belongs to the exam boards. Their joint council defines AI misuse to include "Using AI to complete parts of the assessment so that the work does not reflect the student's own work, analysis, evaluation or calculations", with sanctions up to disqualification and debarment. They name the tools involved under the heading "Homework and subject-specific tools", listing Brainly, Gauth, Photomath, Question AI, Quizlet, Socratic and Symbolab.

So the exam boards have already extended the plagiarism frame from prose to arithmetic. But their rule binds only assessed work. Homework sits in the gap between advisory guidance and enforceable rule, and no official body in the material we reviewed claims otherwise.

The scale of detected misuse is modest and the regulators say so. Penalties for AI-related plagiarism in GCSE, AS and A level rose from around 85 cases in 2024, the first year the category existed, to around 100 in 2025, against 5,025 student malpractice cases in total. Those figures are rounded to the nearest five and count only cases where penalties were applied. The regulator's own annual report pairs the low detected count with "significant concern about the potential scale of undisclosed misuse", particularly "in assessments completed outside controlled conditions".

Both the regulator and the largest US teachers' union agree that detection software does not solve this. The union states that detection tools are "largely inaccurate and unreliable"; the regulator says they should not be treated as sole determinants.

Guidance also disagrees across borders. UNESCO has proposed that AI tools should not be available in classrooms to children under 13. The UK defers to whatever age limit the vendor sets in its own terms of service. US federal guidance sets no age floor, and the Secretary of Education's July 2025 statement said AI "drives personalized learning, sharpens critical thinking", which is close to the opposite of the European Commission's guidelines listing over-reliance among the significant risks.

The Calculator Precedent, Read Properly

Every discussion of this topic reaches the same objection: people panicked about calculators and were wrong.

The objection deserves to be granted, and then read carefully, because what the calculator era actually did is more interesting than either side usually admits.

Three meta-analyses in mathematics education's flagship journal broadly exonerate calculators. Skills were maintained or improved, problem solving improved, attitudes improved. But two findings in that same literature cut the other way. Results on skill retention were not statistically significant, and there was insufficient data to assess transfer.

The calculator consensus was never actually established on the two outcomes that matter most here.

What the system did next is the part worth copying. England's exam regulator did not ban calculators. It wrote a binding condition requiring that "between 33 and 50 per cent of the total marks available" in GCSE mathematics be allocated to questions "which must be completed by Learners without the use of a Calculator". In the United States, NAEP uses the same architecture by design: two thirds of the mathematics assessment is taken without a calculator.

Three features of that solution matter. The duty was placed on the organisation designing the assessment, not on the child. The protection was expressed as a proportion of marks rather than a prohibition. And the regulator kept the binding condition while withdrawing the advisory guidance, which is the exact opposite of how AI is being handled.

The mathematics teaching profession describes the same history. In its February 2024 position statement, the National Council of Teachers of Mathematics notes that in 1980 it said calculators helped students develop problem-solving skills, and that "the outcomes of these tools have led teachers to create assessments that reduce computation and increase problem solving." The response to automated computation was to move the assessment, not to ban the device.

Where the analogy breaks is precise and important. A calculator offloads a sub-step, arithmetic, while the student still owns the reasoning. A chatbot asked "what is the answer" offloads the reasoning itself. A 2026 review in Educational Psychology Review puts the distinction formally: offloading helps when it discharges operations peripheral to the learning goal, and erodes learning when it bypasses the processing that constitutes that goal.

And a calculator is a physical object in a supervised room. The exam rules work because an invigilator is present. There is no non-calculator paper for homework.

What Actually Works Against Over-Reliance on AI in Math Homework

The evidence here is thinner than the alarm, and two findings cut against a simple scare story.

At real-world scale, well-designed AI tutoring mostly did not get used. A two-year randomised trial across 18 middle schools, using a tutor configured to coach rather than answer, produced gains of about 0.06 to 0.08 standard deviations a year, which the researchers note "resemble those from Khan Academy practice without AI assistance". The reason was behavioural: 96% of students tried it once, but the median student messaged it on only a third of practice days, and in just 17% of sessions where they actually made a mistake.

The problem was not that it harmed children. It was that children barely touched it.

Pointed at teachers rather than children, AI does better. A randomised trial of a tool that coached tutors in real time raised topic mastery in math by 4 percentage points, and by 9 points for students of the lowest-rated tutors, at roughly $20 per tutor per year. Message-level analysis showed it increased guiding questions and decreased answer-giving. That trial randomised tutors, not students, and its first author helped build the product into the platform, both of which should be stated.

Carefully engineered AI tutoring can beat strong teaching in a specific setting. A randomised trial with 194 Harvard undergraduates found a large advantage in physics, with the post-test taken without AI. The authors built the tutor and taught the course, it is physics rather than math, and they concede a ceiling effect.

One caution for anyone citing a headline number in the other direction. The most-cited meta-analysis claiming large learning benefits from ChatGPT was retracted in April 2026 over discrepancies that, in the editor's words, "undermine the Editor's confidence in the validity of the analysis". If a draft quotes a pooled figure showing AI boosts learning, it is worth checking it is not that one.

For human tutoring, by contrast, the benchmark is solid: a meta-analysis of 89 randomised trials puts the effect at 0.288 standard deviations, with math at 0.268, and larger when sessions run at least three days a week.

What We Could Not Verify

Naming the gaps is part of the method.

• No study measures children's detection rate for incorrect AI-generated math. The figures in circulation are from university students and adults.

• Photo-math solvers have essentially no credible research. There is no randomised trial, no withdrawal study and no meta-analysis. The circulating effect size traces to a single study of 49 students with no control group. The apps most likely to be in a child's hand are the least studied.

• No source measures how often children use AI for math, only whether they ever have. There is no depth-of-reliance measure for math specifically, and no UK subject-level data at all.

• Nobody measures whether the child understood the answer they submitted.

• No data links an individual child's AI use to their math attainment.

• The exact wording of UNESCO's age recommendation sits inside a document we could not open, so we have attributed the figure to UNESCO's own press release and to its education chief's public statement rather than describing it as a requirement.

• The leading non-retracted meta-analysis on ChatGPT and learning sits behind a publisher block we did not work around, so no pooled effect size appears in this article.

What to Take Away

• Math scores were already falling sharply before generative AI existed, so AI is not the cause of the decline, and any article claiming otherwise is contradicted by its own data.

• The harm from AI in math homework is real but specific: it appears when the tool supplies answers, it shows up only once the tool is removed, and it is driven by copying rather than by wrong answers.

• Children do not notice it happening, which is the most troubling part of the evidence, since students who performed worse did not perceive that they had.

• The risk tracks supervision rather than age, disappearing under proctored conditions and at Grade 5, so the practical lever is when and where a child works, not which year group they are in.

• Spotting machine errors is a teachable skill, and in the one study that tried, four weeks of structured error analysis roughly doubled detection rates.

• If you want to know whether your own child is learning or copying, the test is the one every study here used: take the tool away and see what is left, which is exactly what a free Bhanzu class does by diagnosing what a child can do unaided.

Research Behind This Article

1. Bastani, H. et al., "Generative AI without guardrails can harm learning: Evidence from high school mathematics," PNAS 122(26):e2422633122 (2025). doi.org

2. Liu, G. et al., "AI Assistance Reduces Persistence and Hurts Independent Performance," COLM 2026. arxiv.org

3. Pardos, Z. A. & Bhandari, S., "ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills," PLOS ONE 19(5):e0304013 (2024). doi.org

4. Stadler, M., Bannert, M. & Sailer, M., "Cognitive ease at a cost: LLMs reduce mental effort but compromise depth in student scientific inquiry," Computers in Human Behavior 160:108386 (2024). doi.org

5. Srivatsa, K. A., Maurya, K. K. & Kochmar, E., "LLMs cannot spot math errors, even when allowed to peek into the solution," EMNLP 2025. arxiv.org

6. Mirzadeh, I. et al., "GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models," ICLR 2025. arxiv.org

7. Epoch AI, Capabilities and Benchmarking, benchmark dataset retrieved 27 October 2026. epoch.ai

8. Pew Research Center, How Teens Use and View AI (24 February 2026). pewresearch.org

9. Joung, E., "Building critical GenAI literacy through mathematical error analysis," Frontiers in Education 11 (2026). doi.org

10. Ofqual, GCSE Subject Level Conditions and Requirements for Mathematics, Ofqual/17/6160 (2017). gov.uk

11. National Council of Teachers of Mathematics, Artificial Intelligence and Mathematics Teaching, position statement (February 2024). nctm.org

12. UK Department for Education, Generative artificial intelligence (AI) in education, policy paper (updated 12 August 2025). gov.uk

13. Nakamura, K., "Augmentation or Erosion? Regulating Cognitive Offloading in Generative AI-Supported STEM Learning," Educational Psychology Review (2026). doi.org

14. OECD, PISA 2025 Results (Volume I): Future-Ready Students (September 2026). oecd.org