🔗 Share this article

Technology companies and child protection agencies will receive authority to evaluate whether AI tools can generate child abuse images under new British legislation.

Substantial Rise in AI-Generated Illegal Content

The declaration coincided with revelations from a safety monitoring body showing that cases of AI-generated child sexual abuse material have more than doubled in the past year, rising from 199 in 2024 to 426 in 2025.

New Legal Structure

Under the amendments, the authorities will allow designated AI developers and child protection organizations to examine AI models – the underlying systems for conversational AI and image generators – and ensure they have sufficient protective measures to stop them from creating depictions of child exploitation.

"Ultimately about preventing abuse before it happens," stated Kanishka Narayan, noting: "Experts, under strict conditions, can now detect the danger in AI systems promptly."

Addressing Legal Challenges

The changes have been introduced because it is illegal to produce and own CSAM, meaning that AI creators and others cannot create such images as part of a testing regime. Previously, officials had to delay action until AI-generated CSAM was published online before dealing with it.

This law is aimed at preventing that problem by helping to stop the production of those images at source.

Legislative Structure

The changes are being added by the authorities as revisions to the crime and policing bill, which is also establishing a prohibition on possessing, producing or sharing AI models developed to create child sexual abuse material.

Real-World Impact

This week, the minister toured the London base of a children's helpline and heard a simulated conversation to counsellors involving a report of AI-based abuse. The call portrayed a teenager seeking help after being blackmailed using a sexualised deepfake of himself, created using AI.

"When I learn about children facing extortion online, it is a cause of extreme anger in me and rightful anger amongst parents," he stated.

Concerning Statistics

A prominent internet monitoring organization stated that instances of AI-generated abuse content – such as webpages that may contain numerous images – had significantly increased so far this year.

Cases of the most severe material – the most serious form of abuse – rose from 2,621 images or videos to 3,086.

• Girls were predominantly targeted, accounting for 94% of illegal AI images in 2025
• Portrayals of infants to toddlers increased from five in 2024 to 92 in 2025

Sector Reaction

The law change could "represent a crucial step to ensure AI products are safe before they are released," stated the head of the online safety foundation.

"Artificial intelligence systems have made it so survivors can be targeted repeatedly with just a few clicks, giving criminals the capability to make potentially limitless quantities of advanced, photorealistic child sexual abuse material," she continued. "Content which additionally exploits victims' suffering, and makes young people, especially female children, less safe on and off line."

Counseling Session Information

Childline also released information of counselling interactions where AI has been referenced. AI-related risks discussed in the sessions comprise:

• Employing AI to evaluate weight, body and looks
• Chatbots discouraging children from consulting trusted guardians about abuse
• Being bullied online with AI-generated material
• Online blackmail using AI-faked pictures

Between April and September this year, Childline conducted 367 counselling interactions where AI, conversational AI and related topics were discussed, four times as many as in the same period last year.

Half of the mentions of AI in the 2025 interactions were related to psychological wellbeing and wellness, including using AI assistants for assistance and AI therapeutic applications.