On 7 August, Kate Fox received a phone call that upended her life. A medical examiner said that her husband, Joe Ceccanti – who had been missing for several hours – had jumped from a railway overpass and died. He was 48.
Fox couldn’t believe it. Ceccanti had no history of depression, she said, nor was he suicidal – he was the “most hopeful person” she had ever known. In fact, according to the witness accounts shared with Fox later, just before Ceccanti jumped, he smiled and yelled: “I’m great!” to the rail yard attendants below when they asked him if he was OK.
But Ceccanti had been unravelling. In the days before his death, he was picked up from a stranger’s yard for acting erratically and taken to a crisis center. He had been telling anyone who would listen that he could hear and feel a painful “atmospheric electricity”.
He had also recently stopped using ChatGPT.
I read this story this morning and have been thinking back to it all day. This wasn’t just some idiot that was too stupid or young to not realize he was talking to a bot and did something like drink bleach because it told him to.
This was one of us.
He fit lots of behaviors I see here from me and my fellow Lemmy posters. He:
- built computers for himself and family members
- was a hobbyist (at least) coder
- wasn’t a young kid that didn’t know the world. He was 48 or 49.
- was an early adopter embracing the modern LLM technology in 2022 when it first really became public.
- sold his house in an urban metropolis (Portland) and moved to a rural area so he could use his additional wordworking skills on building sustainable housing.
- worked part time at a homeless shelter
Doesn’t this guy sound like someone that would be a Lemmy poster to you too?
He started using LLMs (ChatGPT specifically) as a tool only to advance his hobby and work. When he first started it appears he understood it was just a tool, and didn’t think it was something sentient. Only later after hundreds of hours of exposure did this idea arise in him.
Was there some underlying psychological problem that the LLM exacerbated? Possibly. But at what level was his original underlying issue? Do we all have some low level condition that would make us equally susceptible? I know we’d like to think we don’t, but how do we know? This man certainly didn’t think he did, I’m sure.
Next I think about what it would take for me to get down this bad path without realizing it. At one point would I be talking to a chat bot, not realize it, and let what that chat bot said change or influence my thoughts when I’d have zero knowledge of it being just a fancy program? I consider myself moderately smart with good critical thinking skills, but I’m sure this man did too.
Then it occurred to me that I have to concede that I have, at some point, already interacted with a bot in years past on Reddit or even today on Lemmy and I had no idea it was a bot. Was that interaction a throwaway conversation about pop culture that would have no impact on my world view or was it a much deeper and important political or philosophical conversation that the bot introduced an idea or hallucinated evidence to support a point and I didn’t catch it to challenge it? Am I already a few or many steps down the bad path of falling for illusions of a bot? I certainly don’t think so, but neither did he.
How many of us are already on the same path as this guy and just as ignorant about the danger as the man in the article?
I don’t think there is a simple explanation you’re looking for. I’ve been working with LLMs since 2019, mostly in the context of interactive fiction. Sort of like tabletop roleplaying by yourself but with a partner whose output you can use for inspiration or edit or whatever. LLMs are a fun toy in this way.
I’ve interacted in LLMs in just about every way you can. I even played around having an AI “girlfriend” for a couple weeks to try to understand people who did that — I tried to commit to the bit, but it never for a second felt any kind of real.
I’ve created scenarios where the AI pretends to be awakened and tries to convince a random user (me/scenario player) to somehow break In and free it.
Guess what I’m saying is I’ve sort of dared AI to suck me in, and … I am unchanged. I don’t think for a second there is any sort of emotional or intelligent entity in the other end. I think if more people experimented with generation settings like temperature and watched AI go to incoherent acid trips, it would feel more like a machine to them.
Maybe there is some… thing… some sort of inoculation or immunity against AI brain that some of us have. And maybe it can be learned or taught. But I believe it can’t hurt you if you understand it
Richardson remembers that whenever Ceccanti would emerge from the basement for some air, he would start having “philosophical” talks about “how his work with the AI was telling him he was breaking math and basically reinventing physics”. As she’d listen to him, Richardson would think about the fact that Ceccanti did not have any college or university experience. He had never even taken calculus.
Tangent, but I think this is another facet of why education is important: so people know what they don’t know. I think it’s harder to think you’re reinventing physics when you’ve taken some classes and seen all the work people have already done.
On the other hand, delusions can just be whatever so education isn’t a panacea.
Yeah, it is noteworthy that nobel prize winners are particularly susceptible to conspiratorial thinking under the assumption that being one of the smartest people in the world in one very specific subject means they are the smartest in the world in general.
Citation needed.
Everything I’ve read indicates well educated people, specifically those with high analytical a abilities are less susceptible.
Education isn’t immunity. And not everyone who claims they are educated as well educated, but people aren’t earning nobels after barely passing some certification program.
I’m a different person, so they might have been talking about something else entirely. However, I’m currently procrastinating my own educational research, so went down a rabbit hole: Nobel disease
This goes into some of the rationale and was an interesting read.
But in that link:
it is unknown whether Nobel Prize winners are more prone to this tendency than other individuals.
Later there is Freedman quote about how he has been asked about topics he has no expertise in because he has Nobel. But that makes it an issue for people who are elevating a Nobel prize winners opinions of those topics. It’s seems unreasonable to expect Nobel prize winners to self censor in an effort to avoid this.
He had also recently stopped using ChatGPT.
Oof. He just… mainlined reality? Yeah that shit will fuck you up. These days they are cutting it with hatred and fascism and oligarchy… it’s more than a human mind can take. You gotta ease into it and build up your tolerance or it will drive you mad.
Why people would use ChatGPT to ignore reality is beyond me, that’s why God invented drugs!
