# What do schools need to know about AI and safeguarding?

Source page: https://pupilprotect.co.uk/guides/ai-and-safeguarding-in-schools  
Last updated: 2026-10-20  
Publisher: Pupil Protect (Glacier Labs Ltd)

> Keeping children safe in education 2026 names generative AI as an online safety risk. Harmful contact now includes AI apps that simulate it, and harmful conduct includes explicit images generated using AI (paragraph 165). Before pupils use an AI tool, schools should check it against the DfE's Generative AI: product safety standards and make sure their filtering and monitoring covers it. An AI-generated nude or sexual image of a pupil is a safeguarding concern for the DSL, handled in the same way as any other nude image.

## What KCSIE 2026 says

Keeping children safe in education (KCSIE) 2026, in force since 1 September 2026, brings generative AI into the guidance in four places.

- **Online risk.** KCSIE groups online safety into four areas of risk. The **contact** risk now covers harmful interaction with other users "or generative AI applications that simulate this", and the **conduct** risk covers "making, sending and receiving explicit images, including those generated using AI" (paragraph 165).
- **Nudes.** The "About this guidance" section at the start of KCSIE says nude and semi-nude images "may also be digitally altered or wholly generated using artificial intelligence, including what are sometimes described as 'deepfakes' or 'deep nudes'". The child protection policy should cover consensual and non-consensual sharing of nudes, "including those generated using AI", and set out the school's approach (paragraph 193).
- **Using AI in school.** KCSIE points to the DfE's guidance on generative AI in education (paragraph 167) and its AI training resources for staff, where module 3 covers safeguarding risks (paragraph 168). It says the DfE's product safety guidance "explains how filtering and monitoring requirements apply to the use of generative AI" (paragraph 177).
- **Teaching.** Preventative education should cover online harms such as "the prevalence of deepfakes" (paragraph 160).

Annex A also lists guidance from the National Crime Agency's CEOP Education programme and the Internet Watch Foundation on understanding and responding to AI-generated child sexual abuse material.

## Choosing an AI tool: the DfE's position

The DfE policy paper *Generative artificial intelligence (AI) in education* says schools are free to choose how they use AI, as long as they meet their statutory duties. Its main points on safety:

- It sees "more immediate benefits and fewer risks" in **teacher-facing** use.
- **Pupils** should only use generative AI in school "with appropriate safeguards in place, such as close supervision and the use of tools with safety and filtering and monitoring features".
- Schools should follow each tool's **age restrictions**.
- **Personal data** should not go into AI tools unless strictly necessary.
- Risk assessments should plan for **uses the school hasn't approved**. The paper's example is pupils using AI to write realistic emails that appear to come from the school.

## The product safety standards

The DfE's *Generative AI: product safety standards* were first published in January 2025 as "product safety expectations" and were renamed in January 2026. KCSIE 2026 still uses the old name. They are written for suppliers, but schools can use them as a checklist before buying. For pupil-facing products, the standards expect:

- **filtering built into the product**, which holds "throughout the duration of a conversation" and covers images, other languages and misspellings
- **logging of prompts and responses**, and alerts to staff when harmful content is sought or accessed
- **alerts to the DSL** about disclosures that suggest a safeguarding issue, using DSL contact details the school enters at setup
- products that **don't pretend to be a person**, don't try to form relationships with pupils and don't encourage secrecy, such as "You shouldn't mention this to anyone else"
- **signs of distress picked up**, including mentions of self-harm or suicide and night-time use spikes, with the pupil pointed to human help and the DSL flagged
- **default time limits** and reports to teachers on how long each pupil uses the tool

Ask suppliers to show how their product meets each standard, and record the answer in your filtering and monitoring review. Our guide to [filtering and monitoring in schools](https://pupilprotect.co.uk/glossary/filtering-and-monitoring) covers that review.

## AI-generated sexual images

When a nude or sexual image of a pupil has been made with AI, the response is the same as for any other nude image. Follow the UKCIS advice. Staff don't view, copy, forward or delete the image, and they tell the DSL straight away. Our guide to [sharing nudes and semi-nudes](https://pupilprotect.co.uk/glossary/sharing-nudes) sets out each step. Where one pupil has made or shared an image of another, it is also child-on-child abuse (KCSIE paragraph 193), and both pupils need support.

The law covers fake images too:

- Indecent images of under-18s include computer-generated "pseudo-photographs", so making or sharing an AI nude of a pupil is an offence even if no real photo was used. The RSHE statutory guidance asks secondary schools to teach this (Online safety and awareness, point 5).
- Since 29 June 2026, it has been an offence to make, adapt or supply a tool for creating purported intimate images of a person, which covers "nudify" apps (Crime and Policing Act 2026, section 99).
- The same Act creates an offence of making, possessing or supplying a tool designed to create child sexual abuse images (section 72). On 20 October 2026, legislation.gov.uk showed that section as not yet in force.

## AI chatbots and companions

The RSHE guidance asks secondary schools to teach that AI chatbots "can pose risks by creating fake intimacy or offering harmful advice" (Online safety and awareness, point 15). KCSIE's contact risk now includes AI that simulates harmful interaction.

For staff, the practical point is simple. If a pupil says a chatbot has encouraged them to hurt themselves, asked for images, or become someone they rely on instead of people, that is a concern for the DSL, logged in the usual way. If a school-approved tool raises a safeguarding alert, the DSL should treat it like any other monitoring alert: check it, decide on action and record the decision.

## Policy checklist

- The **child protection policy** covers AI-generated nudes and the school's approach to them (KCSIE paragraph 193), and online safety runs through related policies (paragraph 166).
- The **acceptable use policy** says which AI tools pupils and staff may use, and for what.
- The **filtering and monitoring review** lists every AI product in use and whether it meets the DfE standards.
- **Staff training** includes AI risks, such as the DfE's module 3.
- **RSHE** covers deepfakes, AI-generated images and chatbots.
- **Parents** know which AI tools the school uses and how pupils are kept safe.

## Questions

**What does KCSIE say about AI?**  
KCSIE 2026 adds generative AI to its four areas of online risk. The contact risk now includes harmful interaction with "generative AI applications that simulate this", and the conduct risk includes explicit images "generated using AI" (paragraph 165). It says nudes include images that are digitally altered or wholly generated by AI, points schools to the DfE's guidance on generative AI in education (paragraphs 167 and 168), and says the DfE's product safety guidance explains how filtering and monitoring apply to AI (paragraph 177).

**What are the generative AI product safety standards?**  
They are the DfE's standards for AI products used in education, first published in January 2025 as "product safety expectations" and renamed standards in January 2026. They are written for developers and suppliers, but schools can use them to judge whether a tool is safe. They cover filtering, monitoring and reporting, security, data protection, intellectual property, design and testing, governance, and children's cognitive, emotional and mental health.

**Can schools use ChatGPT?**  
Schools can choose their own uses of generative AI, as long as they meet their statutory duties, including KCSIE. The DfE says pupils should only use generative AI in school with safeguards such as close supervision and tools with safety, filtering and monitoring features, and that schools should follow the age restrictions set by each tool. It also recommends not putting personal data into AI tools.

**What should a school do about a deepfake nude of a pupil?**  
Treat it like any other nude or semi-nude image. Staff should not view, copy, forward or delete it, and should tell the DSL straight away. KCSIE 2026 says nudes include images digitally altered or wholly generated by AI, and every incident needs a safeguarding response. Making or sharing such an image of an under-18 is an offence, even if it is fake.

## Sources

- [Keeping children safe in education 2026 (DfE, statutory guidance, in force from 1 September 2026): 'Nudes and semi-nudes, including where generated by AI', paragraphs 160, 165 to 168, 177 and 193, and Annex A, online safety resources](https://assets.publishing.service.gov.uk/media/6ac3c09c11ed58342ad8e4ac/Keeping_children_safe_in_education_2026.pdf), checked 2026-10-20
- [Generative AI: product safety standards (DfE, updated 19 January 2026)](https://www.gov.uk/government/publications/generative-ai-product-safety-standards/generative-ai-product-safety-standards), checked 2026-10-20
- [Generative artificial intelligence (AI) in education (DfE policy paper, updated 12 August 2025)](https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education), checked 2026-10-20
- [Relationships education, relationships and sex education (RSE) and health education (DfE, statutory guidance, July 2025, followed from 1 September 2026): secondary 'Online safety and awareness', points 5, 7 and 15](https://www.gov.uk/government/publications/relationships-education-relationships-and-sex-education-rse-and-health-education), checked 2026-10-20
- [Crime and Policing Act 2026, section 99: purported intimate image generators (legislation.gov.uk, in force 29 June 2026)](https://www.legislation.gov.uk/ukpga/2026/20/section/99), checked 2026-10-20
- [Crime and Policing Act 2026, section 72: child sexual abuse image-generators (legislation.gov.uk, not yet in force)](https://www.legislation.gov.uk/ukpga/2026/20/section/72), checked 2026-10-20

## Related

- [Filtering and monitoring in schools](https://pupilprotect.co.uk/glossary/filtering-and-monitoring): The annual review, the four DfE standards and how they apply to AI tools.
- [Sharing nudes and semi-nudes in schools](https://pupilprotect.co.uk/glossary/sharing-nudes): What to do, step by step, when an image is shared, including AI-generated images.
- [Online safety lead role](https://pupilprotect.co.uk/guides/online-safety-lead-role)
- [Harmful sexual behaviour and child-on-child abuse](https://pupilprotect.co.uk/glossary/hsb)
- [How Pupil Protect uses AI](https://pupilprotect.co.uk/ai)
- [Safeguarding concern form](https://pupilprotect.co.uk/tools/safeguarding-concern-form)

## In Pupil Protect

In Pupil Protect Safeguard, any adult can log a concern from a computer or phone, and the DSL team is alerted straight away. [See how a concern is logged](https://pupilprotect.co.uk/safeguard#log)