Social media feeds and chatbots are far from perfect, but they would be entirely unusable without the efforts of people working behind the scenes to purge them of extreme and disturbing content.
Now, as AI models take on more of this work, many of the people who helped train them — often at a personal cost — look set to lose their jobs.
The price of user safety
The work of human content-reviewers is the foundation on which AI safety is built. Data annotators train chatbots directly while moderators judge whether social media posts breach a platform's rules, each making decisions that AI models learn from to filter out harmful content themselves.
Those decisions often come at a psychological price.
Moderators for TikTok, YouTube and Meta platforms have reported being exposed to graphic violence, pornography, self-harm and child sexual abuse, along with beheadings, cannibalism and people being skinned alive.
A former Facebook moderator described the job as "continuous, high-volume exposure to human distress", noting that while casual viewers might quickly scroll past a disturbing video, moderators must "watch the video to the end, and determine exactly where a policy is violated".
At one consultancy, 140 former moderators were diagnosed with "severe" PTSD. At another, some became addicted to graphic content or were pushed towards extreme views by the amount of misinformation and hate speech they read every day.
Annotators for OpenAI's ChatGPT have also been required to classify distressing images and text snippets. An AI trainer for Meta said his work testing safety guardrails had involved spending "days on end pretending to be suicidal or a paedophile".
Tech firms offer varying levels of psychological support. Meta, for example, agreed to provide its moderators with mental health counselling as part of a 2020 legal settlement in the US.
The company also allows its reviewers to blur and remove the colour from images to lessen their impact, though a recent Australian study has questioned the effectiveness of these techniques.
Here come the robots
Having outsourced much of their content review work, pared back "trust and safety" teams and consolidated them into regional hubs, social media companies are now replacing many moderator jobs with AI.
This month, TikTok revealed plans to slash 300 jobs in Dublin, largely centred on trust and safety, as it moves to adopt "the latest technological innovations" (and potentially outsource more jobs via "business partnerships"). It follows multiple rounds of cuts to its Singaporean moderation team.
Meta has said it plans to reduce its "reliance on third-party vendors" for content moderation and to roll out more advanced AI systems to police severe content violations. Earlier this year it announced that, as part of a broader push into AI, the company would shed some 8,000 employees globally.
More than a thousand Dubliners are in the firing line, including 720 from a content-moderation and AI-training contractor. (Again, moderators in Singapore have not been spared.)
Poorer workers at risk
Data annotation work has likewise been largely outsourced, often to poorer nations [$] such as Ghana and India.
The use of contractors in developing nations has allowed companies to deny responsibility for poor work conditions, allegedly including union busting, punishing performance targets and inadequate supports.
In 2023, a Time investigation detailed how Kenyan contractors were paid less than $2 an hour to label some of the most extreme content on which an earlier model of ChatGPT had been trained.
As AI models improve, basic classification workers are increasingly being ditched in favour of specialists. According to the Economist [$], it is now "fairly common" to see job ads for PhD-level annotators.
Meta has already proved itself willing to walk away from entire teams, abruptly terminating a major contract in February this year that left more than 1,000 Kenyan annotators jobless.
We still need human reviewers
While all of this points to less work for human reviewers, these jobs are unlikely to disappear completely.
Meta's transparency reports highlight the mountain of harmful content with which platforms must still contend, along with the shrinking but still-vital role played by people.
In the June 2025 quarter, Meta took down more than 20 million Facebook and Instagram posts for breaches of its "violent and graphic content" policy alone — 97 per cent of it caught before reaching users, likely by AI systems.
The company recently said it would continue to employ human reviewers and use AI to "take on work that's better-suited to technology, like repetitive reviews of graphic content or areas where adversarial actors are constantly changing their tactics, such as with illicit drugs sales or scams".
After all, AI still gets things wrong, particularly when it comes to subtlety and context — such as the difference between sexual content and art, or violent propaganda and news. A "human in the loop" may also be required for legal and reputational reasons.
While it's a good thing that improved AI systems should lead to fewer workers being traumatised, it's also important to recognise how those systems were built. And the companies who benefit from this difficult work must ensure that the people who continue to do it receive adequate support, which could mean bringing more jobs back in-house.
