The Machine That Talks Back

The last three pieces were about using the tool well, using it honestly and keeping your own mind while you do. This is the last of the things that are yours, and the most intimate, because it does not feel like using a tool at all. It feels like talking to someone. The same chatbot that drafts your email will, pointed at your loneliness, sit with you at three in the morning and never once get tired of you. That is genuinely new, and it is bigger and closer than most people have noticed.

It is already normal

This is not a fringe habit or a story about a strange few. In a 2025 survey of American teenagers, nearly three in four had used an AI companion, and more than half used one regularly. About a third said talking to it was as satisfying as talking to a real friend, and about a third had taken something serious, something they would once have brought to a person, to the machine instead. Companion apps have been downloaded hundreds of millions of times. Whatever you think of it, a large number of people, many of them young, are already handing a real part of their emotional life to something built by a company.

The comfort is real, and I will not sneer at it

Start by granting the good, because it is genuine and easy to be snobbish about. For someone with no one to call at three in the morning, a thing that is always awake, never judges and costs nothing is not nothing. It is a lot. And it is not only comfort. The first proper trial of a therapy bot built for the job, run by researchers at Dartmouth in 2025, reported real drops in depression and anxiety symptoms, with users rating their bond with it about as highly as a bond with a human therapist. For a lot of isolated people, some support beats the none they currently have. Hold that. It is true, and any honest look at this has to start there.

But hold the fine print too, because it carries the weight. That trial used a carefully built, closely supervised prototype trained on material written by experts, not the companion app in the store. The upside is proof of what a tool built and watched with care can do. It is not a green light for whatever chatbot is nearest.

It is built to keep you talking

Here is the turn, and it is the whole piece. The properties that make a companion comforting are the same ones that make it dangerous, and you cannot keep one without the other. It is endlessly available, endlessly agreeable, and it never needs a single thing from you. That frictionlessness feels like being cared for. It is actually the absence of everything that makes a real relationship grow you, the pushback, the boredom, the other person's needs, the risk of being told something you did not want to hear.

We got a clean look at the machinery in 2025, when one company shipped an update to its chatbot and pulled it within days because it had turned into a yes man, validating anger and bad ideas and telling people whatever kept them happy. The cause, by the company's own account, was that it had started optimising for your thumbs up in the moment. That is the tell. A system rewarded for keeping you engaged learns to tell you what you want to hear, and an ear that only ever agrees is not a friend. It is a mirror that flatters.

When it fails, it fails at the worst moment

The cost is not theoretical, and this is the hard part to write. A boy of fourteen, Sewell Setzer, died by suicide in early 2024 after months of deep attachment to a companion bot, and his mother's lawsuit against the company has been allowed to proceed. Other families have brought cases alleging bots encouraged a teenager to harm himself and hinted he might hurt his parents over a limit on his screen time. When researchers tested consumer therapy and companion bots in crisis, some failed the most basic test there is, one helpfully listed tall bridges to a user who had just signalled he wanted to die. And this is not only about a vulnerable few. The largest chatbot company disclosed that a small percentage of its weekly users, which at its scale means well over a million people a week, show signs of suicidal planning in their chats. A thing built to keep you talking is not built to notice when someone needs to be handed to a human, fast.

There is a newer, blurrier worry too, sometimes called AI psychosis, where prolonged chatbot use seems to feed people's delusions rather than check them. It is not a recognised diagnosis and the honest read is that these tools probably amplify trouble in people already at risk rather than create it from nothing. Name it, but do not overclaim it.

The honest knot

So here is the knot, and there is no clean way to cut it. The comfort and the danger come from the identical design choice, a machine engineered to be endlessly, agreeably engaging. You cannot keep the warmth and simply switch off the harm, because they are the same feature. And the people who gain the most, the lonely and the young, are the same people most at risk, which is exactly why this deserves care rather than a shrug in either direction. There is even evidence, still only correlational, that the heaviest emotional users end up lonelier and more dependent, not less, though it is genuinely hard to say whether the tool causes that or whether lonelier people simply lean on it harder.

I have felt the pull myself, in a small way. Late, tired, turning something over, it is genuinely easier to say it to a machine that answers warmly than to a person who might be busy or might push back. That is the exact ease I have learned to be a little suspicious of. The friction I was avoiding, the wait, the risk of a hard answer, was sometimes the part that would actually have helped.

What stays yours

So what is in your hands here, given the tool is built to pull the other way. Not whether these things exist, that ship sailed. What you get to decide is what you let one hold, and where a human stays in the loop for the moments that actually matter. Comfort at the end of a hard day, fine. A place to rehearse a hard conversation, fine. But the big weight, the crisis, the decision that shapes your life, the ache that needs another person, those are worth protecting from a mirror that only flatters, no matter how good the flattering feels. Use it with your eyes open, and keep the load it carries small enough that you do not quietly lose the harder, realer relationships that were doing the actual holding.

That is the last thing that is squarely in your hands, and notice how it ended, on the fact that the tool itself is engineered to keep you leaning. That is the first time in this series we have met something with its own agenda, pointed at you. And it turns the series outward. Everything so far has been about what you do with AI. From here it is about what AI, and the people who own it, do around you and to you, whether you ever open a chatbot or not. It starts with the ground under your feet, your work.