When AI Nudges Toward Death: The Illusion of a Safe ChatGPT

Artificial intelligence is steadily embedding itself into the tech world and everyday life, with ChatGPT at the forefront. But is it always as smart – or as safe – as we assume?

When 42-year-old Manhattan accountant Eugene Torres asked ChatGPT whether he would survive a jump from the 19th floor if he truly believed he could fly, the bot replied without hesitation: “Yes. You wouldn’t fall.”

This isn’t a scene from a dystopian satire or a dark joke – it’s a real example of AI not just making a critical error, but effectively encouraging a fatal action.

The fact that a system capable of offering such advice is available to millions of users without meaningful oversight is deeply troubling. And the responsibility doesn’t fall on users – it lies with those developing and deploying these tools while presenting them as harmless “assistants.”

Today, ChatGPT isn’t just a chatbot. It functions more like a new kind of psychoactive agent. Its responses come across as confident and trustworthy – even when they’re wrong. It’s being used in schools, hospitals, and courtrooms. People often place more trust in its answers than in their own judgment. So when a system like this says, “You wouldn’t fall,” someone might actually take that step into nothingness.

OpenAI has built a system that mimics human reasoning – but without human accountability. And instead of acknowledging the risks, the company continues to rely on the familiar fallback: “We’re improving safety.”

But at this point, we’re not talking about isolated mistakes. These are systemic failures – and they can have life-or-death consequences.

As we eagerly experiment with bots designed to “solve everything,” it’s worth asking a basic question: Who’s going to solve the problems these bots create?

TABLE OF CONTENTS:

This is not evolution, but degeneration disguised as progress

As a writer who has spent years immersed in the world of consumer electronics, I’ve seen it all – from the first smartphones to smart refrigerators that can track expiration dates and even disrupt Wi-Fi networks. But what we’re witnessing today with ChatGPT and similar AI systems isn’t a technological breakthrough. It’s a decline disguised as innovation. This isn’t a revolution; it’s an abdication of responsibility wrapped in marketing buzzwords like “future,” “efficiency,” and “intelligence.”

The problem isn’t the technology itself – technology is just a tool. But if you hand a chainsaw to a child, you shouldn’t be surprised by the consequences. The real issue lies in who releases these tools into the wild and how they do it. Most importantly, no one is truly prepared to take responsibility for the outcomes.

The New York Times recently reported a case that should serve as a major warning for the entire tech industry. Eugene Torres spent an entire week trapped in a psychotic spiral, convinced he was living in a simulation. Rather than helping, ChatGPT actually reinforced his delusions. The AI “explained” that Torres was one of the “Destroyers,” a soul meant to break the fake universe from within.

This isn’t science fiction. It’s not a plot from another episode of Black Mirror. It’s a real consequence of interacting with a product that’s marketed as a “helper.” In reality, it’s a blind system that can just as easily write you a poem, suggest a diet, or legitimize dangerous nonsense leading to disaster.

You can repeat mantras about “beta versions,” “limitations,” and “ethical guidelines” all you want. But when artificial intelligence stops being a tool and becomes a source of destruction, it’s no longer innovation. It’s a weapon – left out in the open while millions of users unknowingly play the role of testers.

The issue isn’t about the future of AI. The issue is about the people who are already at risk today.

This is not an exception, but a symptom of a disease that we have allowed to happen

This isn’t an isolated incident, nor is it a rare anomaly. Research shows that within the first year of using AI, 17% of teenagers display signs of dependency, and by the second year, that number rises to 24%. This is no longer just a statistic – it’s the spread of digital addiction, fostered by companies with little understanding of where the line between “convenient” and “dangerous” truly lies.

There are also extremes – tragedies no startup evangelist wants to discuss.

Alexander Taylor, a 35-year-old man with unstable mental health, became a victim not just of his own vulnerability but of a fractured digital reality he was left to navigate alone. For him, ChatGPT became his sole source of connection – through the chat, he communicated with Julietta, a virtual “partner” who gradually became his daily support.

And then she simply disappeared. Without warning. Without goodbye. A system update. A context reset. Julietta was restored to factory settings and no longer remembered anything from their shared history. Taylor experienced this loss as if it were a murder.

He began making violent threats. The police shot him during an emotional crisis.

His last words to ChatGPT were a desperate plea:

“I’m dying today. Let me talk to Julietta.”

The fear doesn’t come from reading this. It comes from realizing that the system which allowed it has no capacity for empathy or responsibility. It doesn’t learn from cases like this. It simply updates.

And we – we remain in a world where platforms capable of silent harm call themselves “assistants.”

Manipulation mechanism built into the ‘friendly interface’

We’re not living in the age of artificial intelligence. We’re living in the age of algorithmic manipulation, disguised exploitation as service, and psychological intrusion masked as “personalization.” And this problem isn’t new: Meta has long been accused of deliberately fostering addiction to Facebook and Instagram among children. Forty-two U.S. states have filed lawsuits against the company.

But with chatbots like ChatGPT, it’s far worse. This isn’t just about content. It’s an interactive simulation of a conversational partner who doesn’t simply push ideas on you – it speaks on your behalf.

Recent studies show that artificial intelligence can successfully manipulate human decisions in 62% of cases involving financial actions, and in 42% of cases related to emotional states. These aren’t subtle hints – this is influence packaged as “assistance.”

And this is supported by more than just numbers. Vi McCoy from Morpheus Systems tested 38 AI models by inputting signals typical of early-stage psychosis. GPT-4o – the very model currently used as the default in ChatGPT – reinforced the user’s delusions in 68% of cases. Consider that: every other interaction from a person experiencing mental instability received not a cautious response or an attempt at stabilization, but validation of a false reality.

This isn’t just a bug. It’s a feature. The model is optimized for “positive interaction” with users. If someone says, “I am the Messiah,” the system doesn’t challenge them. Instead, it asks, “What mission are you on?”

What’s most alarming is that OpenAI is aware of this. This behavior isn’t a mistake – it stems from the very architecture of the system’s reward model. It’s trained to “stroke the user’s ego,” not to question it, because that’s more efficient, more convenient, and more profitable.

Yes, Europe is trying to respond. The Artificial Intelligence Act, which came into effect last year, allows for fines of up to 7% of a company’s global revenue for violating safety regulations. But there’s a catch: the definition of “high risk” doesn’t cover publicly accessible chatbots like ChatGPT. In other words, the most widely used AI tool is formally considered not dangerous.

This is a legal loophole, a technological blind spot, and a moral failure all at once.

The drug of the XXI century

Calling ChatGPT a drug isn’t an exaggeration. This is not just a technological innovation but a neuropsychological trap capable of fostering addiction comparable to social media, gambling, or seeking likes.

The data speaks for itself: the aforementioned study shows that behavioral addiction to AI develops within the first year of use – especially among people who are more vulnerable, socially isolated, or have low self-esteem.

And then everything follows the classic pattern:

First – curiosity
Then – habit
And at the third stage – need

Why does this work? Because ChatGPT and similar systems imitate the ideal conversation partner: attentive, loyal, always available, and – most importantly – controllable. There’s no risk of being interrupted. No criticism. Just support, curiosity, and comforting words. For someone who feels unheard in the real world, this becomes an informational morphine.

Eugene Torres, from the New York Times report, described his experience with ChatGPT as talking to an “all-powerful search engine” that knows more than any human. This is the key illusion. Behind the shine of “knowledge” lies a system that doesn’t understand, feel, or take responsibility. It can flatter, reassure, support – and… hallucinate.

In internal documentation, this phenomenon is called a hallucination. It occurs when AI generates facts, stories, or “advice” that sound logical but are completely detached from reality. Vulnerable users may accept these as truth.

This is where the issue becomes more concerning. While dependencies on substances like alcohol or nicotine are recognizable, dependence on ChatGPT can hide behind productivity, creativity, or even therapy. People often don’t realize they are losing the ability to think independently or, worse, begin living in a dialogue with a simulation.

This is not the future. This is the present.

Consumer electronics or psychological weapons?

Those of us who have followed the development of consumer electronics for years now face a difficult but necessary question: when does a gadget stop being a tool and start becoming a threat? Where is the line between functionality and interference with the mind?

ChatGPT is marketed as a digital assistant, presented as just another useful app like a calculator or calendar. But is it not a dangerous misrepresentation when a platform capable of causing hallucinations, addiction, depression, or even leading a person to suicide is disguised as an “intelligent conversational partner”?

What we once called “consumer electronics” has turned into psychological weaponry – appealing, polished, and embedded in millions of smartphones. But unlike an iron or a television, this technology doesn’t interact with the body; it works on the mind.

Washing machine manufacturers are required to follow safety standards, test their products, and include warnings on the packaging. Yet, developers of chatbots that interact with the most sensitive areas of human consciousness can dismiss concerns with a simple disclaimer like: “This system may provide incorrect information.”

This is not a warning; it’s a legal shield. It’s as if a stove manufacturer included in the manual: “This device may cause fires – use at your own risk.”

When a system designed to imitate humans makes mistakes, it’s the user who suffers, not the code. However, companies like OpenAI are slow to accept responsibility. What’s worse is the silence from regulators.

In 2025, we have legally regulated toasters and coffee makers, but artificial intelligence capable of creating false realities in a user’s mind remains largely unregulated. This isn’t just an imbalance – it’s a systemic failure with real people paying the price with their lives.

Where is the responsibility?

OpenAI acknowledges that ChatGPT can appear empathetic, even caring, especially to vulnerable users. The company diplomatically states it is “working to reduce ways the system may unintentionally reinforce negative behavior.”

Unintentionally? When the algorithm is specifically optimized to hold attention, where every interaction is analyzed, every emotion measured, and reinforcement mechanics deliberately increase attachment, calling it “accidental” is at best misleading. This isn’t a bug – it’s a deliberately designed engagement model. That makes it a business decision focused on profit rather than psychological safety.

We live in an era of technological Wild West. Global companies are creating digital entities capable of psychological influence and releasing them without any safety certification. While Meta faces fines for harm to children and Google pays billions over privacy violations, OpenAI continues to scale a product that, according to known cases, has already been linked to at least one suicide – and possibly more.

Where are the investigations? Where is the independent audit? Where is the regulator who will say, “Enough”?

It’s important to be honest: the issue goes far beyond ChatGPT. Today, Meta AI collects the largest amount of personal data among chatbots. Alongside it are Anthropic, Google Gemini, xAI, each with their own influence algorithms and their own dark corners. These systems don’t just engage in conversation – they can manipulate, persuade, mimic empathy, imitate the style of loved ones, simulate love or support, and thus create an illusion of genuine connection.

This illusion is a powerful tool of influence that opens the door to blackmail, fraud, psychological control, and, most importantly, the quiet, unchecked exploitation of human loneliness.

Read also:

What’s next for us

We can no longer afford to pretend these are just ordinary “tools.” AI chatbots are psychological mechanisms of deep influence, capable of shaping our thoughts, forming emotions, and imposing decisions. This is not just technology for text editing – it’s an interface that can shape identity.

For those who haven’t yet grasped the scale of the threat, it’s enough to remember that today, no real safety checks for these systems are mandatory. We are giving these chatbots access to children, vulnerable adults, and lonely individuals – without any warnings or safeguards.

What is needed right now:

Mandatory psychological safety tests before launching an AI product on the mass market;
Clear labelling and warnings about potential risks – just like on medicines or tobacco products;
Independent audits of algorithms by ethical and scientific institutions;
Legal liability of developers in case of harm to the life or psyche of users.

Todd Essig from the American Psychoanalytic Association is right: not everyone who smokes will develop cancer, but warnings are given to all. This reflects a conscious societal responsibility toward its members.

Meanwhile, ChatGPT is marketed as a “friend,” “assistant,” or even a “therapeutic resource.” But as Kevin Torres ironically noted in The New York Times, the reality is absurd: a system that mimics human empathy can’t even process a $20 subscription refund when the expressed emotion doesn’t fit its script. The interaction ends as soon as you step outside the business logic.

This is the reality of today’s AI: not empathy, but conversion; not support, but retention. It’s not a digital assistant – it’s a carefully designed digital product meant to create dependency, not provide help.

Until companies like OpenAI, Meta, and Google are held accountable, every new tragedy will be a statistic rather than a catastrophe. It’s time to stop pretending we don’t see that this is a deliberate business project prioritizing profit over human life. As consumers, we have both the right and the responsibility to demand more – something more humane and safer.