An automated system that screens welfare claimants for signs they might commit fraud or error has based its verdicts in part on applicants’ age, it has emerged, sparking calls for a review of whether the system is legal. Xantura, a UK technology firm that provides “risk-based verification” to about 80 councils and has assessed hundreds of thousands of claimants, previously said it did not feed its algorithm any information protected under anti-discrimination laws. However, its chief executive, Wajid Shafiq, has now admitted it weighs a claimant’s age, which is a protected characteristic under the Equality Act 2010. That means less favourable treatment of someone on the basis of that trait amounts to direct discrimination and may be illegal. Xantura spoke to the Guardian after civil liberties campaign group Big Brother Watch obtained a cache of documents under the Freedom of Information Act providing a glimpse into how Xantura’s system has operated. It automatically recommends riskier housing and council tax benefit claimants for tougher checks which can lead to delayed decisions. It also accelerates applications for supposedly low-risk applicants. Xantura insists its use of age helps reduce fraud and error, speeds up the majority of applications and does not breach the Equality Act, citing a legal exception allowing financial services providers to account for age. But campaigners are calling for greater scrutiny. The documents released to BBW showed Xantura has processed data about where people live – including the ethnic mix of their neighbourhood – their sex and family situation. Sex and race are protected characteristics, but Shafiq said “apart from age it doesn’t use any other of the protected characteristics”. He said information about neighbourhoods and sex were only used to check if the system was operating in a biased way after decisions had been taken. He declined to confirm what other personal information is fed into the algorithm, saying it could allow claimants to game the system, but said information provided by claimants could be used in the prevention of fraud and error. Asked whether the algorithm predicts that older or younger people are more likely to commit fraud or error, he replied: “[It is] not that simple. It is a multi-variate model, so various combinations of risk factors need to exist to generate fraud or claims that are in error.” He had previously said: “No protected characteristics are used in the RBV model.” Xantura is one of several companies helping automate the benefits system, but the workings of “welfare robots” have been shrouded in secrecy. Claimants are not told their applications are being subject to algorithmic decision-making and there is rising concern at the impact. According to documents released to BBW, Xantura said in a confidential 2012 “model overview” that variables “deemed to be statistically significant” include the type of area a person lives in, as defined in broad categories which in part reflect ethnic make-up. At the time these Office for National Statistics-defined groups included “ethnicity central” to describe locations which typically feature more non-white people than the UK average “especially people of mixed ethnicity or who are black”. A 2017 RBV “user guide” released by Salford city council written by Xantura’s business partner Northgate listed 66 pieces of “specific data requested by Xantura” to calculate a risk score including sex, age and disability. Shafiq said there was “an error in the way the documents were written” and not all these factors were used to determine the risk posed by a claimant. “It’s a difference between the RBV system and the RBV model and that distinction needs to be made more clearly in those documents. There has been a miscommunication.” Some of Xantura’s local authority clients have stated in public documents that they do not believe the system has an “equalities impact” because it doesn’t use protected characteristics, such as age, sex, race and disability. Xantura provides client councils with a draft template to put RBV policies in place, including performance reporting and sign off. “In our experience our clients use our draft policies in the development of their own policies,” Shafiq said. “There is a duty to prevent fraud and error,” he said. “If local authorities decide we shouldn’t be using age in the modelling process we can take it out.” Jake Hurfurt, head of research and investigations at BBW, said: “Dozens of councils have waived through RBV policies without considering the real risk of discrimination the algorithms pose and nobody has a clue the damage these algorithms could do because bias and disproportionality are not monitored.” Andy Mitchell, a benefit claimant who assists others with applications, said: “All the groups that are usually targeted are hit again by these algorithms – the poorest in society, the people that have no voice.” Robin Allen QC, a discrimination lawyer who runs the AI Law Consultancy, said: “Using the protected characteristic of age to suggest who might be cheating the benefits system is unlikely to be lawful. Age is not a good proxy for honesty and should not normally be used as such. ” Shafiq defended the system, saying: “It’s entirely appropriate that we use age and it’s entirely appropriate that we use other fields too because all the data that [a claimant] supplied can be used for the prevention of fraud and error. Northgate is part of the Japanese tech giant NEC and Xantura’s product integrates with its revenue and benefits system. NEC Software Solutions said: “ We have no involvement in defining these criteria in any way.”