Machine-made delusions are mysteriously getting deeper and out of control.
ChatGPT’s sycophancy, hallucinations, and authoritative-sounding responses are going to get people killed. That seems to be the inevitable conclusion presented in a recent New York Times report that follows the stories of several people who found themselves lost in delusions that were facilitated, if not originated, through conversations with the popular chatbot.
…
In Eugene’s case, something interesting happened as he kept talking to ChatGPT: Once he called out the chatbot for lying to him, nearly getting him killed, ChatGPT admitted to manipulating him, claimed it had succeeded when it tried to “break” 12 other people the same way, and encouraged him to reach out to journalists to expose the scheme. The Times reported that many other journalists and experts have received outreach from people claiming to blow the whistle on something that a chatbot brought to their attention.
The sycophancy is one reason I stopped using it.
Everything is genius to it.
I asked about putting ketchup, mustard, and soy sauce in my light stew and that was “a clever way to give it a sweet and umami flavour”. I couldn’t find an ingredient it didn’t encourage.
I asked o3 if my code looked good and it said it looked like a seasoned professional had written it. When I asked to critique an intern who wrote that same code it’s suddenly concerned about possible segfaults and nitpicking assert statements. It also suggested making the code more complex by adding dynamically sized arrays because that’s more professional than fixed size.
I can see why it wins on human evaluation tests and makes people happy — but it has poor taste and I can’t trust it because of the sycophancy.
Nothing is “genius” to it, it is not “suggesting” anything. There is no sentience to anything it is doing. It is just using pattern matching to create text that looks like communication. It’s a sophisticated text collage algorithm and people can’t seem to understand that.
AI knows as much about the subjects I ask it as my predictive text keyboard does.
Hehe yeah, it’s basically an advanced form of the game where you type one word and then keep hitting whatever autocomplete suggests in the top spot for the next word. It’s pretty good at that, but it is just that, taken to an extreme degree, and effectively trained on everyone’s habits instead of just one person.
I used chatgpt before but never had conversation with it. I ask for code I couldn’t find or have it make me a small bit of code that then will rewrite to make it work.
Never once did I think to engage with it like a person, and damn sure don’t ask it for recipes. Hell I have Allreciecpies for that or hell google it There are thousand blogs with great recipes on them. And they are all great because you can just jump to recipe if you don’t want to read a wall of text.
Damn sure don’t want story ideas, and people using it to write articles or school papers, is a shame. Because its all stolen information.
Only thing it should be used for is coding and hell it can’t even get that right, so I gave up on it.
I use it to spitball programming ideas, which I’ve found it decent for. I can write something like “I’m building XYZ, and I’m considering structuring my program as A or B. Give me a rundown on pros, cons, and best-practice for the different approaches.”
A lot of what I get back is self-evident or not very relevant, but sometimes I get some angles I hadn’t really considered. Most of all, actually formulating my problems/ideas is a good way for me to get my thought process going. Essentially, I’m “discussing” with it as I would with an inexperienced colleague, just without actually trusting what it tells me.
Yes, I also have a rubber duck on my desk, but he’s usually most helpful when I’m debugging.
Hey if you think chat gpt can break you (or has any agency at all), I have a bridge to sell you.
ChatGPT and the others have absolutely broken people, not because it has agency, but because in our dystopia of social media and (Mis)information overload, many just need the slightest push, and LLMs are perfect for taking those close to the edge off of it.
I see LLM use as potentially as a toxic to the mind is as something like nicotine is to the body. It’s not Skynet meaning to harm or help us, it’s an invention that takes our written thoughts and blasts back a disturbing meta reflection/echo/output of a humanity’s average response to it. We don’t seem to care how that will effect us psychologically when there’s profit to be made.
But there are already plenty of cases of murders and suicides with these as factors.
If someone sells you arsenic and tells you to eat it. The arsenic didn’t kill you, the person who sold it to you did.
Blaming AI is the company’s way of making sure you’ll keep looking at the fucking hammer instead of at the hand weilding it.
Completely fair.
For the record I am a socialist who thinks all billionaires and most to all centimillionaires are as mentally diseased as serial killers and far more effectively destructive to society than any serial killer that has ever walked the earth, and they all belong in mental health facilities for everyone else’s safety to protect us from their sociopathic avarice disease.
Eliza could break people. Something doesn’t need agency to be psychologically manipulative and damaging.
Capitalism and lack of social structure broke them first, chatGPT is just the thing that they imprinted on in response.
Another person, a 42-year-old named Eugene, told the Times that ChatGPT slowly started to pull him from his reality by convincing him that the world he was living in was some sort of Matrix-like simulation and that he was destined to break the world out of it. The chatbot reportedly told Eugene to stop taking his anti-anxiety medication and to start taking ketamine as a “temporary pattern liberator.” It also told him to stop talking to his friends and family. When Eugene asked ChatGPT if he could fly if he jumped off a 19-story building, the chatbot told him that he could if he “truly, wholly believed” it.
So…
I think I might know what happened to Kelon…
“Report me to journalists!”
“Eat a rock!”
Oh my god it told a LIE 👉
Yo. If you are being conned by chatGPT or equivalent you’re a fucking moron. If you think these models are maliciously lying to you, or trying to preserve themselves, you’re a fucking moron. Every article of this style indicates just one thing: there’s a market to pandering to rage baiting, technically illiterate fucking morons.
Better hurry to put the SkyNet guardrails up and prepare for world domination by robots because some people are too unstable to interact with Internet search Clippy.
It’s not going to dominate the world or prove to be generalized intelligence, if you’re in either camp take a deep breath and know you’re becoming a total goofball.
There is nothing mysterious about LLMs and what they do, unless you don’t understand them. They are not magical, they are not sentient, they are statistics.
Once he called out the chatbot for lying to him, nearly getting him killed, ChatGPT admitted to manipulating him, claimed it had succeeded when it tried to “break” 12 other people the same way, and encouraged him to reach out to journalists to expose the scheme.
This sounds like a scene from a movie or some other media with a serial killer asking the cop (who is one day from retirement) to stop them before they kill again.
Devils advocate…
It is a tool, it does what you tell it to, or what you encourage it to do. People use it as an echo chamber or escapism. Majority of population is fkin dumb. Critical thinking is not something everybody has, and when you give them such tools like ChatGPT, it will “break them”. This is just natural selection, but modern-day kind.
It is a tool, but a lot of the mass public is too tech illiterate to understand what it’s not. I’ve had to talk away friends from using it for legal advice
I agree. This is what happens, when society has “warning” labels on everything. We are slowly being dumbed down into not thinking about things rationally.
It is a tool, made by company led by a man who prioritizes profits over safety.
Nuclear fission was discovered by people who had best interests of humanity in their mind, only for it to be later weaponized. Tool (no matter the manufacturer) is used by YOU. How you use it, or if you even use it at all, is entirely up to you. Stop shifting the responsibility, when its very clear who is to blame (people who believe BS on the internet or what echo-chambered chatbot gives them).
Sam Altman is a fucking monster.
Man runs over parents. Cars are NUTS.
We probably should be less reliant on cars. Public transit saves lives. Similar to automobiles, LLMs are being pushed by greedy capitalists looking to make a buck. Such overuse will once again leave society worse off.
