Sixty minutes. That’s how long it took for a private Snapchat message to travel from Kristen Volpe’s phone to the FBI to the deputies standing in her elementary school hallway. On January 29, 2025, the 22-year-old student teacher at John L. Hensey Elementary in Washington, Illinois, vented to her boyfriend and two roommates — three people total — after a student closed her laptop mid-class and wiped her lesson plan. She referenced shooting the student or the school. She meant it as a frustrated joke. Snapchat’s algorithm didn’t get the humor.
No student reported her. No parent. No coworker. The flag was entirely automated. Snapchat’s policy permits voluntary emergency disclosures to law enforcement — no warrant required — when the platform determines imminent harm exists. The algorithm makes that initial call. Think of it like your Ring doorbell deciding to call 911 on its own, except the doorbell is reading your text messages.
Questions for the ridiculous number of people on here who are perfectly willing to give up their right to privacy because you believe it makes schools and other places safer:
Have you read the 50+ page user agreement for your cell phone company? How about the user agreements of every app that has access to your cell phone, your PC mic or apps that have access to your files? Late model cars have microphones too, as do TV remotes. Microsoft has access to everything on your Windows PC. I’d give 100 to 1 odds that 99% of you haven’t read any of those user agreements OR the UA for private chat apps you are using. Likewise are you sure any cloud storage you’re using isn’t scanning your documents for forbidden phrases?
Since you haven’t read every word of each and every user agreement and think it keeps schools safe, you should be just fine with the fact that EVERYTHING you say or write may be available to law enforcement, even if you’re talking to yourself while sitting on the toilet in your home. If you mutter that you’d like to kill someone or burn something misheard as a school building to the ground you are just fine with AI flagging it and the police knocking on your door in an hour, right? You’ll also be just fine with losing your career and your job never being able to find another one, right? Right? After all, other people never know WTF someone dangerous and violent is going to do.
The number of people on here who are perfectly willing to give up all their rights and have their private communications monitored for something that is nothing more than security theater is unbelievable.
Very well said! Privacy protects the innocent far more than it protects the guilty, and no false positive rate other than zero is acceptable when ‘van bursts in to take your career away over an AI detecting wrongdoing’ in my view.
If a false positive rate of zero is not possible then not building it is the only ethical option.
My gawd. People are missing the issue here. You described it pretty well OP.
Yes there are weirdos and they must be caught. Algorithms are getting out of hand though, targeting their wrong people and not the weirdos. There is a massive privacy issue here. Must every text that we send be accessible by the government? I don’t buy the argument of “I have nothing to hide”. When you are in the hot seat they will use any info they can get their hands on. make an offside joke they will use it.
Fuck this shit.
Even people who have “nothing to hide” aren’t safe.
I’ve lost track of the number of articles about people who were wrongly accused by “AI” camera systems or flagged for arrest because their (relatively common-seeming) name matches or is close to an entirely different person who is a legitimate suspect. Some of those people have been jailed for weeks and have lost their jobs over what amounts to a clerical error.
Real US judges have ruled that click-wrapped ToSes are not legally binding, since they commonly are bypassed, and app developers use dark patterns to discourage end users from reading them.
Also, ToSes and EULAs are notoriously verbose and difficult to read. People with college level reading skills struggle with them.
So no, blaming the end user is not appropriate here.
That’s a bigger deal than you may think it is, given that US college students are testing dumber and dumber every year, and is accelerating thanks to AI.