“It’s much easier to build an AI system that can detect a nipple than it is to determine what is linguistically hate speech.” The Facebook founder Mark Zuckerberg made that comment in 2018 when he was discussing how the company tackles content that is deemed inappropriate or, in Facebook terms, judged to be violating community standards. Facebook’s artificial intelligence technology for identifying nudity gets it right more often than not. Between January and March this year, Facebook removed 39.5m pieces of content for adult nudity or sexual activity, and 99.2% of it was removed automatically, without a user reporting it. There were 2.5m appeals against removal and 613,000 pieces of content were restored. But it doesn’t work every time, and the AI has problems with historical photos and paintings. As Guardian Australia reported this week, a user was suspended for posting an 1890s image of Aboriginal men in chains in response to the Australian prime minister’s claim – on which he subsequently backtracked – that Australia didn’t have slavery. Facebook acknowledged blocking the image post was an error and restored it, but the Guardian story about the block was also prevented from being posted – as was the subsequent story about the first story being blocked. We received dozens of emails from readers who were prevented from posting the stories, or were even temporarily banned from Facebook for trying to post them. The image itself was whitelisted but Facebook’s systems did not apply the same whitelisting to the sharing of articles featuring that image, leading to the seemingly endless cycle of penalties. It was an AI error but for people attempting to share the image or stories it seemed as though Facebook was taking an incorrect, hardline position on a particular issue while allowing other posts – including the US president Donald Trump’s inflammatory posts – to remain untouched. There’s no doubt Facebook has a moderation problem which it is, in part, trying to automate its way out of. Given the horrific stories of the third-party moderators suffering post-traumatic stress disorder from having to review content on Facebook all day – for which tens of thousands are now seeking compensation – it is no surprise Facebook is trying to automate everything. Facebook is already using AI to moderate its platform for nudity to the point where users are appealing few decisions, but when it comes to hate speech, misinformation and other content the AI is still a work in progress. Zuckerberg’s point about the “nipple” is that it is much easier for systems to recognise and repeatedly block an image without human intervention than it is for a similar AI system to analyse the text of a post or a message in context and determine it is hate speech. The AI will make mistakes but the hope is that it will eventually learn. It will be interesting to see what errors Facebook makes in applying the image-scanning AI to its quest to limit the amount of misinformation being spread on its platform in relation to the Covid-19 pandemic. In April the company put misinformation labels on 50m posts related to Covid-19, based on factchecks of 7,500 articles. It also removed 2.5m products such as face masks and hand sanitiser from Marketplace since 1 March. Once a claim has been factchecked and deemed misinformation, the image used in the original post is scanned so it is picked up in the future when people try to share it. Identical images have the factchecking label attached to it, whether those images link to articles or not. But a cursory glance at any one of the groups still promoting Covid-related conspiracy theories reveals that a lot of misinformation is slipping through the cracks. There is as yet no evidence that Facebook is “over-censoring” misinformation, regardless of what some might claim in these groups. The activist and journalist Cory Doctorow raised the point this week that Facebook is increasingly being asked to take on more content moderation tasks but he said it might be the wrong request. A platform so big it struggles to determine what is or isn’t appropriate content across multiple cultures may be better reduced to a smaller scale, so that standards match the communities it operates in, and there is a transparent review process that doesn’t require intervention by media or public pressure to remove something – or in this case, restore something. Where Facebook sees AI as the solution to keeping up, Doctorow suggests AI failures shows the task is simply too big for Facebook or anyone else, and Facebook should be cut down to a “size, to a scale where communities can set and enforce norms”.