INNOVATE Issue 7_2025 | Page 13

The limits of current AI detection strategies

Detecting AI-generated work is far more difficult than many teachers realise. In 2023, TeacherTapp reported that only 6 % of teachers believed students were using AI without permission. A year later, that number jumped to 20 %, suggesting growing awareness. However, this also likely underestimates the true extent of AI use.

For example, Fleckenstein et al.( 2024) found that teachers correctly identified AI-generated work only 38 % of the time. Worse still, they were often overconfident in their abilities, a phenomenon we call the“ Detection Overconfidence Problem.”

How well you think you can detect AI

How well you can actually detect AI

Detection overconfidence

While teachers familiar with their students’ writing styles may have a slight advantage, this still leaves a massive gap between perception and reality. Clearly, relying on instinct alone is not an effective strategy for identifying AI use in student work.

To overcome this, many teachers have sought out AI-detection tools. Unfortunately, even the best AI detection software is only about 67 % accurate( Weber- Wulff et al., 2023), meaning that many students are either wrongly accused or slipping through the cracks. The authors were blunt in their assessment: these tools are“ neither accurate nor reliable.”

This puts both students and teachers in a difficult position: students may face false accusations, while teachers cannot confidently determine whether work is original or AI-assisted. Until detection methods improve, schools need to consider alternative approaches, such as teaching students how to use AI responsibly rather than focusing solely on preventing its use.

AI and feedback

AI-generated feedback has the potential to support student learning, but it still falls short of what teachers can provide. A recent study from the University of California compared human and AI-generated feedback on secondary school essays, evaluating them across five key criteria( Steiss et al., 2024). While AI slightly outperformed teachers in providing criteria-based feedback, teachers were better in four out of five categories, particularly in accuracy and prioritisation.

Notably, AI struggled with high-quality essays, sometimes misidentifying areas for improvement. This highlights a major limitation – AI feedback, while quick and structured, often lacks the nuanced understanding and contextual awareness that experienced educators bring to the table.

Despite its limitations, AI can still play a useful role in the feedback process. For example, AI can also deliver time-consuming personalised feedback by analysing student performance over time. Another one of its strengths is providing fast, detailed, and objective evaluations based on set criteria( Meincke et al., 2024). Specifically, AI can highlight grammatical errors, suggest structural improvements, and offer targeted recommendations, potentially saving teachers valuable time.

However, its weaknesses cannot be ignored. AI is prone to“ hallucinations,” meaning it can generate incorrect or misleading information with unwarranted confidence. It also lacks the emotional intelligence to encourage students effectively( Ruwe & Mayweg-Paus, 2023). While AI should not replace teacher feedback, it can be a helpful tool for early drafts, offering students timely suggestions for revision before receiving more in-depth guidance from their teachers. The key is ensuring that both students and educators understand AI’ s strengths and limitations, using it as a complement rather than a substitute for human expertise.