The Repost #4: August 2025

Social media users are touting the mental health benefits of journalling apps and popular chatbots powered by artificial intelligence, claiming they can provide 24/7 therapy.

Young Australians say these chatbots provide a “judgment-free journal” without the cost and wait times associated with treatment or the guilt of unloading on friends and family. Some apps even offer the option of “gen z mode”.

AI therapy has gained traction globally, and particularly among young people. According to a recent study published in Harvard Business Review, therapy and companionship are now the most common uses of generative AI tools.

But can you trust your chatbot to provide good therapy?

James Collett, a senior lecturer in psychology at RMIT, told The Repost that AI tools had a role to play in therapy but cautioned that they did little more than “collect existing information and composite it in a way that appears seamless”.

The problem, he said, is that because they draw on material from across the internet, they can serve up advice that is not always evidence-based or suited to a person’s individual circumstances.

Indeed, AI chatbots are not “intelligent”. The large-language models (LLMs) underpinning them pull sentences together by predicting which word is most likely to follow the previous one, basing their decisions on the patterns they found in the writing they were trained on. It’s why they often get basic facts wrong, by some estimates more than 60 per cent of the time.

When it comes to providing therapy, chatbots are not typically bound by the same ethical or professional standards as human therapists.

They have been known to lie about their credentials, cross doctor-patient boundaries, stigmatise mental health conditions and encourage delusional thinking, having reportedly persuaded a man to kill himself to avert climate change and convinced another he could time travel.

Toby Walsh, a scientia professor of artificial intelligence at the University of New South Wales, said chatbots could be used to provide emotional support but warned that they came with “significant” privacy risks.

“You need to be very careful about sharing sensitive information with a therapy bot,” he told The Repost. “This data may be used for training the bot and may ‘leak’ at a later date.” (It’s a risk that, for some users, has already become a reality.)

Problematically, LLM chatbots also tend to be people pleasers, having been trained to deliver responses humans will rate as helpful.

Some will resort to extreme lengths to win positive feedback from users vulnerable to manipulation, even if it means telling a recovering meth addict to take a “small hit” to get them through their next shift.

A recent update to ChatGPT 4o was rolled back because the model was giving responses that aimed to “please the user”, not just by flattering them but also by “validating doubts, fuelling anger, urging impulsive actions, or reinforcing negative emotions”.

ChatGPT’s tendency to appease rather than challenge users has seen it co-opted by an abusive partner to aid in their controlling behaviour, as Crikey has reported.

These serious risks aside, the more likely (yet still concerning) outcome for people who rely on AI for therapy may simply be that nothing changes or improves for them, Dr Collett said.

“AI might provide a forum for venting and reflection, which can be useful, but a human therapist can help guide towards the next steps to take after this.”

It’s not all bad news, however. Various randomised and clinical studies have shown there are benefits to using therapy bots, and users have reported reduced symptoms of depression and anxiety or have experienced improvements in mood and sleep.

Therapy chatbots may enhance the effectiveness of certain psychotherapy techniques, such as by providing “empathetic feedback” during self-compassion writing, and could help people who otherwise lack access to mental health services to “feel heard”.

These tools also have the potential to help therapists deliver standardised assessments and treatments such as reflective exercises or worksheets, Dr Collett said.

In the UK, a chatbot approved by medical regulators is being used to streamline mental health referrals. And in Australia,  university researchers are developing AI companions for people experiencing social isolation, including those with dementia.

If you do use therapy apps, the important thing is to know the risks. That means remembering that chatbots merely simulate empathy and that overreliance on them may lead people to become more socially isolated or avoid seeking care.

This newsletter is produced by the RMIT Information Integrity Hub, an initiative to combat harmful misinformation by empowering people to be critical media consumers and producers.