Pupil Protect

Guides › AI and safeguarding in schools

What do schools need to know about AI and safeguarding?

Keeping children safe in education 2026 names generative AI as an online safety risk. Harmful contact now includes AI apps that simulate it, and harmful conduct includes explicit images generated using AI (paragraph 165). Before pupils use an AI tool, schools should check it against the DfE's Generative AI: product safety standards and make sure their filtering and monitoring covers it. An AI-generated nude or sexual image of a pupil is a safeguarding concern for the DSL, handled in the same way as any other nude image.

Updated . Checked against the official guidance. Sources

What KCSIE 2026 says

Keeping children safe in education (KCSIE) 2026, in force since 1 September 2026, brings generative AI into the guidance in four places.

  • Online risk. KCSIE groups online safety into four areas of risk. The contact risk now covers harmful interaction with other users “or generative AI applications that simulate this”, and the conduct risk covers “making, sending and receiving explicit images, including those generated using AI” (paragraph 165).
  • Nudes. The “About this guidance” section at the start of KCSIE says nude and semi-nude images “may also be digitally altered or wholly generated using artificial intelligence, including what are sometimes described as ‘deepfakes’ or ‘deep nudes’”. The child protection policy should cover consensual and non-consensual sharing of nudes, “including those generated using AI”, and set out the school’s approach (paragraph 193).
  • Using AI in school. KCSIE points to the DfE’s guidance on generative AI in education (paragraph 167) and its AI training resources for staff, where module 3 covers safeguarding risks (paragraph 168). It says the DfE’s product safety guidance “explains how filtering and monitoring requirements apply to the use of generative AI” (paragraph 177).
  • Teaching. Preventative education should cover online harms such as “the prevalence of deepfakes” (paragraph 160).

Annex A also lists guidance from the National Crime Agency’s CEOP Education programme and the Internet Watch Foundation on understanding and responding to AI-generated child sexual abuse material.

Choosing an AI tool: the DfE’s position

The DfE policy paper Generative artificial intelligence (AI) in education says schools are free to choose how they use AI, as long as they meet their statutory duties. Its main points on safety:

  • It sees “more immediate benefits and fewer risks” in teacher-facing use.
  • Pupils should only use generative AI in school “with appropriate safeguards in place, such as close supervision and the use of tools with safety and filtering and monitoring features”.
  • Schools should follow each tool’s age restrictions.
  • Personal data should not go into AI tools unless strictly necessary.
  • Risk assessments should plan for uses the school hasn’t approved. The paper’s example is pupils using AI to write realistic emails that appear to come from the school.

The product safety standards

The DfE’s Generative AI: product safety standards were first published in January 2025 as “product safety expectations” and were renamed in January 2026. KCSIE 2026 still uses the old name. They are written for suppliers, but schools can use them as a checklist before buying. For pupil-facing products, the standards expect:

  • filtering built into the product, which holds “throughout the duration of a conversation” and covers images, other languages and misspellings
  • logging of prompts and responses, and alerts to staff when harmful content is sought or accessed
  • alerts to the DSL about disclosures that suggest a safeguarding issue, using DSL contact details the school enters at setup
  • products that don’t pretend to be a person, don’t try to form relationships with pupils and don’t encourage secrecy, such as “You shouldn’t mention this to anyone else”
  • signs of distress picked up, including mentions of self-harm or suicide and night-time use spikes, with the pupil pointed to human help and the DSL flagged
  • default time limits and reports to teachers on how long each pupil uses the tool

Ask suppliers to show how their product meets each standard, and record the answer in your filtering and monitoring review. Our guide to filtering and monitoring in schools covers that review.

AI-generated sexual images

When a nude or sexual image of a pupil has been made with AI, the response is the same as for any other nude image. Follow the UKCIS advice. Staff don’t view, copy, forward or delete the image, and they tell the DSL straight away. Our guide to sharing nudes and semi-nudes sets out each step. Where one pupil has made or shared an image of another, it is also child-on-child abuse (KCSIE paragraph 193), and both pupils need support.

The law covers fake images too:

  • Indecent images of under-18s include computer-generated “pseudo-photographs”, so making or sharing an AI nude of a pupil is an offence even if no real photo was used. The RSHE statutory guidance asks secondary schools to teach this (Online safety and awareness, point 5).
  • Since 29 June 2026, it has been an offence to make, adapt or supply a tool for creating purported intimate images of a person, which covers “nudify” apps (Crime and Policing Act 2026, section 99).
  • The same Act creates an offence of making, possessing or supplying a tool designed to create child sexual abuse images (section 72). On 20 October 2026, legislation.gov.uk showed that section as not yet in force.

AI chatbots and companions

The RSHE guidance asks secondary schools to teach that AI chatbots “can pose risks by creating fake intimacy or offering harmful advice” (Online safety and awareness, point 15). KCSIE’s contact risk now includes AI that simulates harmful interaction.

For staff, the practical point is simple. If a pupil says a chatbot has encouraged them to hurt themselves, asked for images, or become someone they rely on instead of people, that is a concern for the DSL, logged in the usual way. If a school-approved tool raises a safeguarding alert, the DSL should treat it like any other monitoring alert: check it, decide on action and record the decision.

Policy checklist

  • The child protection policy covers AI-generated nudes and the school’s approach to them (KCSIE paragraph 193), and online safety runs through related policies (paragraph 166).
  • The acceptable use policy says which AI tools pupils and staff may use, and for what.
  • The filtering and monitoring review lists every AI product in use and whether it meets the DfE standards.
  • Staff training includes AI risks, such as the DfE’s module 3.
  • RSHE covers deepfakes, AI-generated images and chatbots.
  • Parents know which AI tools the school uses and how pupils are kept safe.

Common questions

What does KCSIE say about AI?

KCSIE 2026 adds generative AI to its four areas of online risk. The contact risk now includes harmful interaction with "generative AI applications that simulate this", and the conduct risk includes explicit images "generated using AI" (paragraph 165). It says nudes include images that are digitally altered or wholly generated by AI, points schools to the DfE's guidance on generative AI in education (paragraphs 167 and 168), and says the DfE's product safety guidance explains how filtering and monitoring apply to AI (paragraph 177).

What are the generative AI product safety standards?

They are the DfE's standards for AI products used in education, first published in January 2025 as "product safety expectations" and renamed standards in January 2026. They are written for developers and suppliers, but schools can use them to judge whether a tool is safe. They cover filtering, monitoring and reporting, security, data protection, intellectual property, design and testing, governance, and children's cognitive, emotional and mental health.

Can schools use ChatGPT?

Schools can choose their own uses of generative AI, as long as they meet their statutory duties, including KCSIE. The DfE says pupils should only use generative AI in school with safeguards such as close supervision and tools with safety, filtering and monitoring features, and that schools should follow the age restrictions set by each tool. It also recommends not putting personal data into AI tools.

What should a school do about a deepfake nude of a pupil?

Treat it like any other nude or semi-nude image. Staff should not view, copy, forward or delete it, and should tell the DSL straight away. KCSIE 2026 says nudes include images digitally altered or wholly generated by AI, and every incident needs a safeguarding response. Making or sharing such an image of an under-18 is an offence, even if it is fake.

Related

All guides