Captured source text (L5)
The technical accuracy criterion accuracy criterion?
Ensure the method(s) has been evaluated against appropriate metrics and the
results indicate that the method(s) is able to correctly establish whether or not a
particular user is a child
4.11To understand the technical accuracy of an age assurance method, service providers should
ensure it has been evaluated against appropriate metrics to assess the extent to which they
can correctly determine the age or age range of a person under test lab conditions.
4.12Age assurance methods either produce:
• A binary result (for example, categorising users as either over or under the age of 18).
• A continuous result (for example, providing an estimation of the user’s age).17
4.13In the case of methods that produce a binary result, examples of appropriate metrics
include but are not limited to; the True Positive Rate (TPR), False Positive Rate (FPR), and
False Negative Rate (FNR).18
4.14In the case of methods that produce a continuous result, examples of appropriate metrics
include but are not limited to the Standard Deviation, Mean Absolute Percentage Error
(MAPE), and Cumulative Score (CS).19
4.15Service providers should be satisfied that the results indicate that the age assurance
method(s) is able to correctly establish whether or not a particular user is a child.
Use a challenge age approach for age estimation methods
4.16Where the age assurance process used on the service involves the use of age estimation,
the provider should use a challenge age approach.
4.17A challenge age approach is widely used offline when selling age-restricted products in
retail environments, for instance, through the retailing strategy ‘Challenge 25.’ In this
approach, anyone who appears to the provider of restricted products to be under the age
of 25 should be challenged to provide acceptable ID proving that they are over the age of
18 if they wish to buy alcohol. The ‘challenge age’ in this scenario would be 25.20
4.18In an online age assurance process, a challenge age approach refers to where a user who is
estimated as being under a given challenge age must then undergo a second age assurance
step (for example, a different age assurance method) to confirm that they are over the
required age.21
4.19The challenge age should be set at an appropriate level according to the limits of the
technical accuracy of that method, for example, where system testing suggests that there is
a significant risk of incorrectly estimating a 17-year-old’s age by 7 years above or below. To
17 The estimation of the user’s age will usually be accompanied by a confidence interval or range, which
conveys the algorithm’s level of uncertainty regarding the prediction. For example, where an age estimation
method predicts that a user is 25 years old with a confidence interval of ±2 years, this means that the method
estimates the user’s age to fall within the range of 23 to 27 years.
18 We define each of the metrics set out in the technical glossary in Annex 1 of this document.
19 We define each of the metrics set out in the technical glossary in Annex 1 of this document.
20 Drink Aware, Challenge 25.
21 ACCS, 2022. Measurement of Age Assurance Technologies.
13
manage this risk a buffer can be set above the age by 8 years, so if the relevant age is 18
then the Challenge Age would be 25. For users estimated to be over the age of 25, no
additional verification will be required. Where the method estimates that the user’s age is
under the challenge age, the user could be required to undergo another age check by a
second method that is more technically accurate for that age group.
4.20Using a challenge age approach helps to improve the overall effectiveness of the age
assurance process by preventing or minimising borderline cases where the age estimation
method incorrectly assesses a user as being an adult when they are a child.
Periodically review the technical accuracy of the age assurance method(s) and
make changes where necessary
Full captured verbatim text
Ensure the method(s) has been evaluated against appropriate metrics and the results indicate that the method(s) is able to correctly establish whether or not a particular user is a child 4.11 To understand the technical accuracy of an age assurance method, service providers should ensure it has been evaluated against appropriate metrics to assess the extent to which they can correctly determine the age or age range of a person under test lab conditions. 4.12 Age assurance methods either produce: • A binary result (for example, categorising users as either over or under the age of 18). • A continuous result (for example, providing an estimation of the user’s age).17 4.13 In the case of methods that produce a binary result, examples of appropriate metrics include but are not limited to; the True Positive Rate (TPR), False Positive Rate (FPR), and False Negative Rate (FNR).18 4.14 In the case of methods that produce a continuous result, examples of appropriate metrics include but are not limited to the Standard Deviation, Mean Absolute Percentage Error (MAPE), and Cumulative Score (CS).19 4.15 Service providers should be satisfied that the results indicate that the age assurance method(s) is able to correctly establish whether or not a particular user is a child. Use a challenge age approach for age estimation methods 4.16 Where the age assurance process used on the service involves the use of age estimation, the provider should use a challenge age approach. 4.17 A challenge age approach is widely used offline when selling age-restricted products in retail environments, for instance, through the retailing strategy ‘Challenge 25.’ In this approach, anyone who appears to the provider of restricted products to be under the age of 25 should be challenged to provide acceptable ID proving that they are over the age of 18 if they wish to buy alcohol. The ‘challenge age’ in this scenario would be 25.20 4.18 In an online age assurance process, a challenge age approach refers to where a user who is estimated as being under a given challenge age must then undergo a second age assurance step (for example, a different age assurance method) to confirm that they are over the required age.21 4.19 The challenge age should be set at an appropriate level according to the limits of the technical accuracy of that method, for example, where system testing suggests that there is a significant risk of incorrectly estimating a 17-year-old’s age by 7 years above or below. To 17 The estimation of the user’s age will usually be accompanied by a confidence interval or range, which conveys the algorithm’s level of uncertainty regarding the prediction. For example, where an age estimation method predicts that a user is 25 years old with a confidence interval of ±2 years, this means that the method estimates the user’s age to fall within the range of 23 to 27 years. 18 We define each of the metrics set out in the technical glossary in Annex 1 of this document. 19 We define each of the metrics set out in the technical glossary in Annex 1 of this document. 20 Drink Aware, Challenge 25. 21 ACCS, 2022. Measurement of Age Assurance Technologies. 13 manage this risk a buffer can be set above the age by 8 years, so if the relevant age is 18 then the Challenge Age would be 25. For users estimated to be over the age of 25, no additional verification will be required. Where the method estimates that the user’s age is under the challenge age, the user could be required to undergo another age check by a second method that is more technically accurate for that age group. 4.20 Using a challenge age approach helps to improve the overall effectiveness of the age assurance process by preventing or minimising borderline cases where the age estimation method incorrectly assesses a user as being an adult when they are a child. Periodically review the technical accuracy of the age assurance method(s) and make changes where necessary