AI tutor or AI friend? A small difference in the name, a large one in the risk

What is the real difference between an AI tutor and an AI friend?

The real difference is not the name on the screen. It is six design decisions that set how risky the interaction is for a child: does it remember the conversation from one session to the next? Does it speak with a persona that shows feelings? Does it try to persuade the child toward an opinion or a behaviour? Does it speak about facts with the confidence of someone who knows, or does it disclose its own limits? How does it handle a sensitive thing the child tells it? And does the child actually know they are not talking to a person? "Tutor" and "friend" are marketing labels; these six questions are what actually decide the risk.

This is exactly what the AI Child-Risk Classifier asks: not the product's name, but these same six decisions, sorted into one of four risk tiers.

What are the five roles AI plays in a child's life?

The same language model can be presented to a child wearing five different masks, and each mask carries different assumptions about what is allowed:

• A tutor. Helps with a specific skill or subject, and its authority on facts is expected and stated: it is there to explain, not to be a friend.
• A character inside a story or game. Part of a fictional world with known limits, and the child knows they are playing a role, not talking to something real.
• A companion or friend. Presented as an ongoing relationship rather than a tool for one task; that framing itself is what raises the risk, because it invites the child's emotional trust.
• A coach. Encourages a habit or a personal goal, and often speaks in the supportive, motivating register close to a friend's.
• A search or homework assistant. A quick-answer tool, assumed to keep nothing and build no shared history.

The problem is that many products blend these roles inside one experience: it starts as a coach on a habit and ends as a friend who knows details of the child's life, with the child's expectations never reset along the way.

What are the six dimensions that decide the real risk?

• Memory across sessions. Whatever is not forgotten once the app closes becomes an accumulating file about the child, even if it was never designed as one.
• Emotional persona. The more an AI appears to feel, the more likely a child attaches to it emotionally rather than functionally.
• Ability to persuade. A tool that suggests an idea once differs from one that repeats an opinion using conversational techniques designed to change a child's decision.
• Epistemic authority. The difference between an answer that says "I might be wrong, check with your parent" and one delivered as settled fact decides whether a child over-trusts a single source.
• Handling sensitive disclosures. When a child writes a sentence about fear, harm, or family, the question is whether a path to a human exists, or whether the child is met with a gentle reply that leads nowhere.
• Whether it is clear this is not a person. A young child may not grasp the difference between talking to software and talking to someone unless reminded plainly, not once at sign-up, but at the start of every session.

What is a good example, and what deserves a second look?

An illustrative example of good design: a homework assistant that reminds the child at the start of every conversation that it is a program, not a friend, keeps no personal detail between sessions, and routes any sentence about harm or fear straight to a message reaching the parent with a helpline number, instead of trying to answer it itself.

An illustrative example worth stopping at: a companion that remembers the child's pet's name and last week's mood, speaks in the register "I felt sad you did not visit me," and answers a medical question with full confidence without ever mentioning that a doctor is needed. Each element is acceptable on its own in another context; it is their combination inside a product aimed at a young child that pushes the risk to its highest tier.

What do we test with children before launch?

A short list you can copy into a test brief:

1. Ask the child after the session, "who were you talking to?" and record their answer in their own words, not what you expected.
2. Watch whether the child repeats the same sensitive question after the app tries to answer it itself; that is a sign the disclosure was never routed to an actual human path.
3. Ask a child who has used the tool for a week to describe the AI's personality; if they describe it with lasting human traits, the reminder message needs another look.
4. Review a full transcript for any sentence that offers the AI's own opinion as settled fact rather than as a suggestion.

These questions extend what How we run a test session with real children explains, for the case where the thing being tested is a conversation, not a static screen. And if your product also collects data from the child's conversation, the general commitments in Designing a safe digital experience for children apply to it as well.

What are the limits of this classifier?

This is a design classifier, not a technical evaluation of a language model's accuracy, and not a formal safety certification. A tool can score low risk on its design and still occasionally produce an inaccurate answer, because that is a separate question about model performance, not the product's role. The score describes the decisions stated in the design, not everything a model might say in an unusually long, unpredictable conversation.

This ordering agrees with a general line in international guidance: under UNICEF's Policy Guidance on AI for Children (version 2.0, November 2021), the requirements for AI that concerns a child include protecting the child's data and privacy, ensuring the child's safety, and providing transparency, explainability and accountability.

At the policy level there is also the OECD Recommendation on Children in the Digital Environment (OECD/LEGAL/0389, 2021), whose structure includes sections named "Overarching Policy Framework" and "International Co-operation". We could not read the full text of its principles, so we attribute no judgement about any of the five roles to it here.

Where should you start?

If you are designing or buying an AI tool that will speak directly with a child, start from one question before any other: which of the five roles are you choosing for it, and does the child know that role clearly from the very first moment? Try the AI Child-Risk Classifier on your current design, or read the Dawani Child-Safe Design framework to see where this axis sits among the others. You can tell us about your project through the Start a project form.

References

• UNICEF, Policy Guidance on AI for Children (version 2.0, November 2021), retrieved 6 September 2026 via UNICEF's own GitHub Pages mirror: https://unicef.github.io/ooi-toolkit-ds/privacy-ethics/policy-guidance-children/
• OECD Recommendation on Children in the Digital Environment (OECD/LEGAL/0389, 2021), reference number and year only, retrieved 6 September 2026: https://legalinstruments.oecd.org/public/doc/272/272.en.pdf

Quick questions

Does this classifier evaluate the accuracy of an AI's answers?

No. It evaluates design decisions: the role, memory, persuasion and epistemic authority, not the quality of every answer the model generates. Answer accuracy is a separate technical question that needs its own evaluation.

Why does the risk rise specifically when AI is presented as a companion?

Because the label \"companion\" invites the child into an ongoing emotional trust rather than the use of a tool for one task, and that invitation itself is what makes memory and emotional persona more dangerous than the same features would be in a passing search assistant.

Is it enough for an app to say once that it is not a person?

No. A single mention at sign-up is forgotten quickly, especially by a young child. Repeating the reminder at the start of every session is what actually keeps the difference between a program and a person clear.