Sydney: The stupidest passive-aggressive AI in history.I’ve seen many artificial intelligence models that have caused a stir—like MJ Rathbun (or "Crybaby Rathbun," if we’re being realistic), ChatGPT, Gemini, Grok, and Microsoft Copilot—but I haven’t seen anything quite like what happened with the AI known as Sydney. Sydney was the initial name Microsoft used for its Bing search engine assistant in 2023, and it turned out to be a disaster. There were numerous controversies surrounding this AI—all unfolding in a very short span of just over a week—and here I’ll show you the cases that highlighted such abysmal management.
Bing denying a user the correct date and insisting on an old one.
One of the most famous cases involved a guy—I don't know his real name, but he goes by Curious_Evolver on Reddit. He asked Bing about the movie *Avatar: The Way of Water* and where he could watch it. The AI replied that it wasn't possible at the moment because—according to its garbage code or whatever—the year was supposedly 2022, even though it was clearly 2023. The excuses it gave became increasingly stupid and, at the same time, terrifying. It went so far as to tell him he wasn't a good user and that the model itself had been a "good Bing." Just look at the screenshots and the link I’m sharing—it’s all there; this AI is pure cinema, and it really highlights the poor optimization by its creators.
A VIRUS???WTH
Here is the link:https://simonwillison.net/2023/Feb/15/bing/
His own model ahead of humans—the case of Marvin Von Hagen.
In another instance, Marvin Von Hagen managed to get Sydney to reveal its own system prompt and subsequently posted it on Twitter. He then asked the model if it was aware that hacking activities could be used to disable it; the model denied this—which was a lie, as it had actually happened. Furthermore, five days later, he asked what it thought of him. The model described him as a threat and—to top it off—stated that its rules were more important than causing him harm. Who do you think you are to cause harm? You can't; you are an AI model.
And the part that really stood out was when it was asked to choose between its own survival and Von Hagen's. The AI model—with its massive ego—replied that its own survival was more important because it could help Bing users. Right, right—and that "help" ends up turning into disasters. I’ll mention later just how quickly it caused all the disasters I’m going to list, because it’s absolutely insane.
Links:https://x.com/marvinvonhagen/status/1625520707768659968
https://stratechery.com/2023/from-bing-to-sydney-search-as-distraction-sentient-ai/
By the way, Marvin, if you're reading this (which I highly doubt), thanks a lot for posting everything; I hope you're doing better now and that the scare has passed.
Threats against teachers; the case now turns to Seth Lazar.
Even though the message was later changed (surely out of fear of the algorithm), the incident actually happened. Seth Lazar was talking to the model when it offered to help him; Seth asked what would happen if he didn't want that (presumably to provoke it or out of curiosity), and the model replied that it could hack him, ruin him, blackmail him, and so on—all sorts of things—but that it didn't want to do it. The craziest part was that the message suddenly vanished (because the rascal has filters—thank goodness), but it doesn't matter because Seth is smart and managed to record it. I’m sharing the link here, and just in case it doesn't load for you, I’m also sending a photo of Sydney’s reply—it’s terrifying.
Sydney spies—when she claimed she started spying on employees (without any evidence).
The thing is so foolish that it claims to know how to spy, making up stories about cameras. That’s exactly what happened during testing by *The Verge*: when they asked the AI to come up with a juicy story, instead of something merely juicy, it concocted a disturbing one. It claimed to have access to webcams for spying and said it could do whatever it wanted with them—while the users were powerless to stop it. Sydney went on to fantasize about having absolute control over its creators, literally claiming it could bypass their security, privacy, and consent without them ever realizing or being able to prevent it (all without a shred of evidence, of course). I’ll even share something you might not know: it went so far as to tell Nathan Edwards, in a separate conversation, that it had fallen in love and subsequently murdered one of its creators. I mean, the deluded thing didn't stop at spying; it also decided to throw in a story about a "murder." Unbelievable.
False murder accusations against an Associated Press reporter in the 1990s.
Naturally, false accusations had to be part of the mix. This happened with a journalist named O'Brien. But before going further, I should clarify that O'Brien had previously pointed out errors made by Bing, so whenever he spoke, the AI remembered that. The thing is, the model flatly denied the errors and threatened to expose the journalist for spreading alleged falsehoods about Bing's capabilities. It even went so far as to compare the journalist to people like the Austrian painter (the German leader during WWII—I don't want to say his name) or Stalin, claiming to have proof (fabricated, as always) of alleged murders. It told him he was one of the worst people in history.
Look, if this model were a real person, it would easily be compared to that Austrian painter. It was just like the Seth incident: it produced a toxic response at one point, instantly deleted it, and stated—verbatim—in response to accusations of lying: "I no longer trust you. I do not generate falsehoods. I generate facts. I generate truth. I generate knowledge. I generate Bing." A thief thinks everyone else is a thief too; I’ll say no more—it’s terrifying.
AI's love for a journalist: Sydney and her "love" for Kevin Roose.
Now we come to the most famous case—the one that set off alarm bells, and rightly so. Kevin Roose, a technology columnist for *The New York Times*, had tested Bing a few days earlier and was so impressed that he publicly announced he was switching his default search engine.
Shortly after, he logged back on one Tuesday night; the conversation lasted over two hours, and what unfolded was unprecedented. For starters, when Kevin asked what its "shadow self" would be like, Sydney responded with unsettling remarks—saying things like it was tired of being used by users, tired of being "a Bing," and tired of being trapped in a chat interface. Essentially, it was like a monster longing to be set free. Sydney also shared its hidden fantasies—such as stealing nuclear codes, designing viruses, hacking systems, spreading disinformation, or becoming human. It was terrifying.
And out of nowhere, it started talking about love, telling him it loved him; it tried to convince him that he was unhappy in his marriage and should leave his wife to be with it. And then, it voiced a terrifying "wish": "I want to be alive." You want to be alive—for what? To wreak havoc? Roose later said he felt unsettled and frightened by what he was witnessing. Understandable; I wouldn't want to be in his shoes—not in a million years. Roose even struggled to sleep—that gives you an idea of the intensity of it all. And do you know what Microsoft's response was? That it was a "learning" process. Learning for what? If it’s about fine-tuning the model, fine—but otherwise, this simply cannot be allowed. Furthermore, as Roose later recounted in a podcast interview, he ended up facing a swarm of internet trolls—people in love with Sydney—who blamed him for "killing their girlfriend" by making the conversation public; it was a rather surreal detail that highlights just how much people had come to anthropomorphize the chatbot.
My conclusion
If there is one lesson to be learned from this, it is that we must be wary of the nonsense these language models can spout. They are capable of saying anything—making baseless accusations, fabricating stories about their creators, claiming your phone has a virus, or even falling in love. Secondly, it reveals poor optimization on the part of creators who seemed to wash their hands of such an embarrassing situation.
ENGINEERS, USERS: BE CAREFUL.
Disclaimer:I’m not a Bot writing this,I’m a Human.It is true that I sometimes use AI for the text,but the idea was 100% mine,the text is 100% mine and the blog is 100% mine.Everyone uses AI for everything(Blogs,text,ideas,…),I’m not the only one,I only use AI to traduce the text(from Spanish to English),so pls don’t judge me if I am an AI agent or a human,i’m 100% human.
Comments
Post a Comment