Measuring impact with adolescents: what may honestly be claimed?

What may honestly be claimed about a program's impact on adolescents?

You may claim what was actually measured, correctly typed: proximal indicators that move during the program, not promises of a changed life. An honest claim states its comparison, its guardrail indicator, and the limits of how far it generalizes. Anything else is marketing dressed up as numbers.

Measuring impact with adolescents is harder than with younger children, for two opposite reasons. An adolescent can fill out any questionnaire with whatever pleases the person asking, and can just as easily boycott any instrument that smells of paternalism. Between pleasing and boycotting, many numbers that look precise on the surface quietly dissolve. A younger child usually answers with what they think is correct in an adult's eyes, but rarely refuses to take part at all. An adolescent has the option of a silent withdrawal, and uses it the moment a tool feels like surveillance rather than understanding.

What is the difference between a proximal and a distal indicator?

We separate two layers of indicators. Proximal indicators move during the program itself and can reasonably be attributed to it: an intended behaviour that recurs, a skill demonstrated in a specific task, a self-report carefully designed rather than borrowed from an off-the-shelf tool. Distal indicators, such as overall academic attainment or mental health over years, are shaped by dozens of simultaneous forces, and cannot honestly be attributed to a single program without a rigorous research design rarely available outside a university.

The common error in this field measures the proximal and then claims the distal: use of a time-management tool rises, and the program announces improved mental health for participants. We commit to the reverse: we measure the proximal and claim it alone, naming the distal as a hoped-for direction rather than a proven result. The difference worth discussing between a group that took part and one that did not is the difference of the two differences, before the program and after it, not one rising line that any number of other causes could explain just as well.

Take a hypothetical example to make the difference concrete: a program aims to improve an adolescent's relationship with their screen time. The honest proximal indicator here is how often the adolescent uses a self-set time-management tool, or their own report of how they felt after a specific session. Claiming the program "improved overall mental health" or "raised academic attainment" is a distal claim that no single proximal indicator supports, however logical the link looks on paper.

What indicators respect the adolescent?

Three conditions we hold to in any measurement tool for this age group:

• Voluntariness before volume. The share who keep participating without being required to is more honest than the total number of sign-ups. Voluntary continuation is a headline indicator in our evaluation, not a footnote. A program that requires attendance conflates schedule discipline with a participant's actual conviction, and those are entirely different things when we ask what really worked.
• Guardrail indicators are mandatory. Every program carries a possible harm that must be watched by name rather than left to chance: usage hidden instead of reduced, anxiety rising in the name of discipline, or a sense of surveillance eroding trust inside the family. A report with no guardrail indicator is itself a warning sign, whatever the rest of its numbers look like.
• Self-report is designed, not translated. Few questions, in a register close to how an adolescent actually speaks, timed not to interrupt another activity, with real anonymity stated plainly. A long questionnaire translated literally from a foreign tool measures an adolescent's patience for answering, not their actual state.

How do you honestly phrase a claim?

We bind ourselves to typing every results sentence correctly before it is published: is it observed, measured, or interpretation? An observed sentence describes what the team saw with no formal measurement tool. A measured sentence names its instrument, its time window, and its comparison; a hypothetical sentence shaped like "60% of participants use the tool regularly after one month, against 35% in the comparison group" is a template for a fully formed measured sentence, whatever program it happens to describe. An interpretation is declared as one, plainly, and never disguised as a measured number. This three-way distinction is simple to describe and hard to keep to day to day, and it is the same thinking behind how we talk about the launch stage in our process, where impact is measured against the goal the work started from, not against the number of minutes a teenager spent inside the experience.

Where should you start?

If you are building a program for adolescents and need a measurement frame that holds up against a real teenager, not only a review committee, start from one question: what will we actually measure, not what we would like to say a year from now. Write the proximal indicator down on its own, separate from the distal goal, and if you cannot state it clearly, that is a sign the design itself, not only the measurement, needs another look. Tell us what you need through the Start a project form, or read our short answers to questions that come up often.

References

• Ryan, R. M., & Deci, E. L. (2000). "Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being." American Psychologist, 55(1), 68-78. Self-determination theory, a standard academic reference with no link.
• UN Committee on the Rights of the Child, General Comment No. 25 (2021) on children's rights in relation to the digital environment, CRC/C/GC/25, including adolescents' participation in the design of what concerns them, their measurement tools included. An official reference named without a link.

Quick questions

What is a guardrail indicator, and why is it mandatory?

It is an indicator that watches one specific possible harm, such as usage being hidden rather than reduced, or anxiety rising in the name of discipline. An impact report with no guardrail indicator hides the place where a program could be going wrong.

Why isn't it enough for an adolescent to say they enjoyed the program?

Because satisfaction is easy to measure in the moment and does not by itself indicate a lasting change in behaviour or skill. Satisfaction is a useful indicator alongside others, not a substitute for measuring what actually changed.

When is it honest to talk about long-term impact?

When a research design compares a group that took part with one that did not, over a period long enough to show a real difference. Without that comparison, distal impact stays a hoped-for direction, not a proven result.