Investigation will explore possibility of human biases being reinforced by AI systems
The government’s data ethics centre and the Cabinet Office’s Race Disparity Unit will launch a joint investigation into the potential for bias in algorithmic decision-making, it has been announced.
The Centre for Data Ethics and Innovation will examine how human biases can be reflected in algorithms that are being increasingly used in the criminal justice system, local government, recruitment and financial services can reflect human bias, it said in a strategy document published this week.
The strategy sets out CDEI’s priorities for the next two years, and is the first such document to be published since its board met for the first time in December. The strategy also sets out how the centre plans to engage with the public to help people better understand how data-driven technology is used, in the hope of creating a “trusted and trustworthy environment for innovation”.
These algorithms are intended to rule out unconscious bias, but there is growing concern about the potential for developers to inadvertently embed their own biases into algorithms they create. There have been reports of algorithms used to screen CVs displaying gender bias, for example.
Related content
- Bursting the bubble – the ethics of political campaigning in an algorithmic age
- Public-sector AI code of conduct published
- How the police wants to use AI and analytics to ‘adopt a public-health approach to crime’
The Race Disparity Unit, which analyses government data on the experiences of people from different ethnic backgrounds, will work with CDEI on the project to explore the potential for bias based on ethnicity in decisions made in the criminal justice system. Together, the two bodies will come up with a set of recommendations for government.
In its announcement of the investigation, the government said there was scope for algorithms to be used to assess the likelihood of reoffending and inform decisions about policing, probation and parole.
Some police forces have already started using algorithms to inform decisions, it said, including one that is using them to help assess the risk of someone reoffending, and therefore to decide whether they should qualify for deferred prosecution.
The science and technology select committee of MPs is among the groups that have raised concerns about how people might fall foul of biases in decisions made using algorithms. In a report in May last year, it called for government to give citizens a legal “right to explanation” which would mean they were entitled to know how decisions affecting them had been made.
In its response in September, the government acknowledged that the civil service needed to be transparent about how it used algorithms, although it did not commit to producing a list of where algorithms “with significant impacts” were used, as recommended by the committee.
In a statement accompanying the report’s publication, digital secretary Jeremy Wright said CDEI would help to ensure that technology used to improve people’s lives would be developed “in a safe and secure way”.
“I’m pleased its team of experts is undertaking an investigation into the potential for bias in algorithmic decision-making in areas including crime, justice and financial services. I look forward to seeing the centre’s recommendations to government on any action we need to take to help make sure we maximise the benefits of these powerful technologies for society.”
The centre, which was first announced in the November 2017 Budget, is funded by the Department for Digital, Culture, Media and Sport but is independent from government. It is currently acting as an expert committee, but the government has said it will give it statutory powers in future.
Last month the chair of the CDEI said bias would be one of the main topics the centre would focus on in its first few months, along with micro-targeting in advertising.
Speaking at the Public Sector AI summit hosted by PublicTechnology last month, Roger Taylor said CDEI would examine how best to test AI systems to ensure that the public “believe that the governance of the system is in line with societal values”.
In addition to bias, Taylor picked out micro-targeting as the other key area on which CDEI will focus its initial efforts.
