Captured source text (L5)
The reliability criterion implementing age assurance?
Ensure that methods with a degree of variance have been suitably tested and
that ongoing performance is measured and monitored
4.43Age assurance methods that rely on machine learning or artificial intelligence, such as facial
age estimation and photo-ID matching, are likely to produce outputs with a degree of
variance. There are several reasons for this, including data variability and model complexity.
4.44In addition, the performance of these specific methods may degrade over time due to
‘model drift’. This is where the data the method has been trained on becomes less
representative of the population using the age assurance method. For example, population
demographics may shift over time, resulting in a greater degree of variance.
4.45Where service providers implement age assurance methods that rely on artificial
intelligence or machine learning, we expect them to take steps to ensure that it has been
suitably tested during the development process to ensure it produces reproducible results,
i.e. to ensure that outputs are consistently produced when the method is presented with
the same inputs.
4.46There should be regular monitoring and measurement of key performance indicators of the
system once the age assurance process has been deployed.27 Where necessary, root cause
analysis and retraining should also be carried out where unexpected or unreliable
predictions are being observed, particularly where such predictions may risk children being
able to access harmful content.
4.47For other kinds of age assurance methods, including credit card age checks, open banking,
and MNO age checks, outputs do not generally exhibit any variance of the type described
above. In these cases, identical outputs should be produced when the method is presented
with the same inputs. While reliability is less relevant for these methods, the service
provider should still ensure that they fulfil the criteria of technical accuracy, robustness, and
fairness.
Ensure that the evidence used is derived from a trustworthy source
4.48We expect service providers to have confidence in the evidence that the age assurance
method is relying on by considering, for example:
• the nature and properties of any identity documents, profiles, accounts, data, etc. used
as part of the age assurance process; and
• the source of the underlying data or documents.
27 For example:
1) Age Verification Accuracy Rate (AVAR): the percentage of users correctly identified as belonging to
the appropriate age group;
2) Age Verification Efficiency (AVE): the time taken to complete the age verification process;
3) Drift Threshold: establish predefined thresholds for AVAR and AVE beyond which significant model
drifting is considered to have occurred.
17
4.49In assessing the nature and properties of the relevant evidence, service providers should
identify features that they would expect to see in a trustworthy source. When deploying
photo-ID matching, for example, these features might include that:
• the evidence has originated from a country or organisation that is recognised as
trustworthy;
• the positioning of the photographs on the evidence does not suggest they have been
edited or replaced;
• the layout or any logos look as expected; and/or
• the visible security features are genuine.28
4.50Certification against the trust framework indicates that the evidence used by a third-party
digital identity or attribute service provider should be reliable.29
Full captured verbatim text
Ensure that methods with a degree of variance have been suitably tested and that ongoing performance is measured and monitored 4.43 Age assurance methods that rely on machine learning or artificial intelligence, such as facial age estimation and photo-ID matching, are likely to produce outputs with a degree of variance. There are several reasons for this, including data variability and model complexity. 4.44 In addition, the performance of these specific methods may degrade over time due to ‘model drift’. This is where the data the method has been trained on becomes less representative of the population using the age assurance method. For example, population demographics may shift over time, resulting in a greater degree of variance. 4.45 Where service providers implement age assurance methods that rely on artificial intelligence or machine learning, we expect them to take steps to ensure that it has been suitably tested during the development process to ensure it produces reproducible results, i.e. to ensure that outputs are consistently produced when the method is presented with the same inputs. 4.46 There should be regular monitoring and measurement of key performance indicators of the system once the age assurance process has been deployed.27 Where necessary, root cause analysis and retraining should also be carried out where unexpected or unreliable predictions are being observed, particularly where such predictions may risk children being able to access harmful content. 4.47 For other kinds of age assurance methods, including credit card age checks, open banking, and MNO age checks, outputs do not generally exhibit any variance of the type described above. In these cases, identical outputs should be produced when the method is presented with the same inputs. While reliability is less relevant for these methods, the service provider should still ensure that they fulfil the criteria of technical accuracy, robustness, and fairness. Ensure that the evidence used is derived from a trustworthy source 4.48 We expect service providers to have confidence in the evidence that the age assurance method is relying on by considering, for example: • the nature and properties of any identity documents, profiles, accounts, data, etc. used as part of the age assurance process; and • the source of the underlying data or documents. 27 For example: 1) Age Verification Accuracy Rate (AVAR): the percentage of users correctly identified as belonging to the appropriate age group; 2) Age Verification Efficiency (AVE): the time taken to complete the age verification process; 3) Drift Threshold: establish predefined thresholds for AVAR and AVE beyond which significant model drifting is considered to have occurred. 17 4.49 In assessing the nature and properties of the relevant evidence, service providers should identify features that they would expect to see in a trustworthy source. When deploying photo-ID matching, for example, these features might include that: • the evidence has originated from a country or organisation that is recognised as trustworthy; • the positioning of the photographs on the evidence does not suggest they have been edited or replaced; • the layout or any logos look as expected; and/or • the visible security features are genuine.28 4.50 Certification against the trust framework indicates that the evidence used by a third-party digital identity or attribute service provider should be reliable.29