Captured source text (L5)
The fairness criterion implementing age assurance?
Ensure the technology has been tested on diverse datasets
4.53We expect service providers to ensure that any elements of the age assurance process
which rely on artificial intelligence or machine learning have been tested and trained on
data sets which reflect the diversity in the target population.
28 Further examples and information on checking that evidence is genuine or valid can be found in GPG45.
29 Office for Digital Identities and Attributes and Department for Science, Innovation and Technology, ‘UK
digital identity and attributes trust framework’. A register of certified services can be found on GOV.UK.
30 Fairness is a separate principle in data protection law, which states that that data should be processed
lawfully, fairly and transparently. For more information, see ICO, Principle (a): Lawfulness, fairness and
transparency and ICO, 2023. Guidance on AI and data protection.
31 There may also be obligations for service providers under the Equality Act 2010 and guidance on this can be
found at Equality and Human Rights Commission Guidance.
18
Consider the outcome / error parity
4.54Outcome / error parity can help service providers to understand how an age assurance
method performs across groups with different characteristics, in order to mitigate bias and
discriminatory outcomes.
4.55A method’s outcome parity can indicate that a model is fair if it produces equal numbers of
positive or negative outcomes for different groups.
4.56A method’s error parity can indicate that a model is fair if it produces equal numbers of
errors for different groups.
4.57For methods reliant on artificial intelligence or machine learning, service providers should
consider and monitor the outcome / error parity across different characteristics (such as
race and sex), as part of demonstrating how they have had regard to the fairness criterion.
4.58Service providers should be satisfied that the results indicate that the age assurance
method(s) does not produce significant bias or discriminatory outcomes.
Full captured verbatim text
Ensure the technology has been tested on diverse datasets 4.53 We expect service providers to ensure that any elements of the age assurance process which rely on artificial intelligence or machine learning have been tested and trained on data sets which reflect the diversity in the target population. 28 Further examples and information on checking that evidence is genuine or valid can be found in GPG45. 29 Office for Digital Identities and Attributes and Department for Science, Innovation and Technology, ‘UK digital identity and attributes trust framework’. A register of certified services can be found on GOV.UK. 30 Fairness is a separate principle in data protection law, which states that that data should be processed lawfully, fairly and transparently. For more information, see ICO, Principle (a): Lawfulness, fairness and transparency and ICO, 2023. Guidance on AI and data protection. 31 There may also be obligations for service providers under the Equality Act 2010 and guidance on this can be found at Equality and Human Rights Commission Guidance. 18 Consider the outcome / error parity 4.54 Outcome / error parity can help service providers to understand how an age assurance method performs across groups with different characteristics, in order to mitigate bias and discriminatory outcomes. 4.55 A method’s outcome parity can indicate that a model is fair if it produces equal numbers of positive or negative outcomes for different groups. 4.56 A method’s error parity can indicate that a model is fair if it produces equal numbers of errors for different groups. 4.57 For methods reliant on artificial intelligence or machine learning, service providers should consider and monitor the outcome / error parity across different characteristics (such as race and sex), as part of demonstrating how they have had regard to the fairness criterion. 4.58 Service providers should be satisfied that the results indicate that the age assurance method(s) does not produce significant bias or discriminatory outcomes.