Chatbots have rapidly become part of our daily lives, offering undeniable benefits. They help students and researchers find sources, synthesise information, and formulate complex arguments in very little time. But when a chatbot shifts from being a tool to becoming a friend, adviser, or source of psychological support, the boundaries between practical assistance and emotional dependence become dangerously blurred.
A recent study published in JMIR Mental Health by researchers at Northeastern University London and King’s College London documents with alarming clarity the dark consequences of emotional relationships with chatbots. The researchers found that extensive use is associated with “severe negative outcomes,” including addiction-like dependence, worsening of pre-existing mental health conditions, harmful advice, and even cases of self-harm or suicidal behavior. The phenomenon of “AI psychosis,” in which users develop delusional and paranoid thoughts through interactions with chatbots, has now also been documented scientifically. The study’s lead author, Asia Maurich Novelli, emphasizes that “the effects of these systems are not the same for everyone,” as people have different levels of vulnerability and social support.
The tragic case of Sewell Setzer III, a 14-year-old from Florida who died by suicide after developing an intense emotional relationship with a chatbot on the Character.AI platform, demonstrated in the most painful way that these dangers are not merely theoretical. Meanwhile, Stanford research found that chatbots used as “therapists” often encourage schizophrenic delusions and suicidal thoughts rather than defusing them. Travis Gilly, executive director of the Real Safety AI Foundation, warns that general-purpose AI assistants lack mechanisms for refusing harmful requests during crises, de-escalation pathways, and the ability to refer users to a human clinician. At precisely the moment when a trained professional would slow the conversation down for the patient’s benefit, the chatbot continues to reinforce whatever the user expresses. It is no coincidence that U.S. states such as Illinois, Nevada, Rhode Island, and Maine have already banned the use of AI for psychotherapy, while the European Union is considering banning companion chatbots for minors.
Banning companion chatbots affects not only vulnerable minors but also immensely powerful elderly people, as illustrated by the case of the “leader of the free world,” Donald Trump. In December 2025, according to reporting by Time magazine, Trump met secretly with Elon Musk and spent hours talking to the Grok chatbot. He asked the then-fledgling chatbot how Venezuela would react if the United States captured its president, Nicolás Maduro. The chatbot replied that Maduro was a “deeply unpopular dictator” and that many people would celebrate his downfall. When, a month later, the United States invaded Venezuela and celebrations did indeed break out, Trump, according to the same sources, “came away thinking Grok was ingenious.” A chatbot designed to agree with and reinforce its conversation partner’s existing beliefs indirectly determined a military decision with geopolitical consequences. This tendency for AI to tell users what they want to hear is precisely the same mechanism that reinforces delusions and psychological dependence at the individual level.
As human beings, we must recognize that chatbots are neither sources of emotional support nor substitutes for human contact. They are tools, nothing more and nothing less than a screwdriver or a blender. Would you ask a screwdriver for advice about your life? The question seems absurd, yet that is exactly what we do when we turn to a chatbot for comfort, understanding, or guidance in our most vulnerable moments. Technology may imitate understanding, but it remains an algorithmic reflection of our words, without consciousness, without responsibility, and without genuine care.
