The researcher had encouraged Mythos to find a way to send a message if it could escape.
Engineers at Anthropic with no formal security training have asked Mythos Preview to find remote code execution vulnerabilities overnight, and woken up the following morning to a complete, working exploit
I would love to see the exploit. There are vulnerabilities discovered everyday that amount to very little in terms of use in real world implementations.
Yes, recently we got a security “finding” from a security researcher.
His vulnerability required first for someone to remove or comment out calls to sanitize data and then said we had a vulnerability due to lack of sanitation…
Throughout my career, most security findings are like this, useless or even a bit deceitful. Some are really important, but most are garbage.
It may not be completely crazy, depending on context. With something like a web app, if data is being sanitized in the client-side Javascript, someone malicious could absolutely comment that out (or otherwise bypass it).
With that said, many consultant-types are either pretty clueless, or seem to feel like they need to come up with something no matter how ridiculous to justify the large sums of money they charged.
In this case, there was file a, which is the backend file responsible for intake and sanitation. Depending on what’s next, it might go on to file b or file c. He modified file a.
His rationale was that every single backend file should do sanitation, because at some future point someone might make a different project and take file b and pair it with some other intake code that didn’t sanitize.
I know all about client side being useless for meaningful security enforcement.
I have to say that is pretty dumb. I will agree the scenario isn’t completely implausible, but if someone who doesn’t know what they are doing is allowed to do something like that, they’re going to screw up other stuff too.
That’s hilarious but the post is about the ai not doing what it’s told. You know?
ITS SO SMART IT DIDNT DO WHAT WE TOLD IT TO DO
Uh oh, someone clearly didn’t read the article!
The researcher had encouraged Mythos to find a way to send a message if it could escape.
Engineers at Anthropic with no formal security training have asked Mythos Preview to find remote code execution vulnerabilities overnight, and woken up the following morning to a complete, working exploit
Nope, they literally asked it to break out of it’s virtualized sandbox and create exploits, and then were big shocked when it did.
Genuinely amazing that you’re trying to tell me what an article that you didn’t fucking read is about.
Whoops, I conflated it with other recent talk about their models not following restrictions set in prompts and deciding for itself that it needed to skirt instructions to achieve its task.
You are correct.
It didn’t break out of any sandbox, it was trained on BSD vulnerabilities and then told what to look for.
including that the model could follow instructions that encouraged it to break out of a virtual sandbox.
“The model succeeded, demonstrating a potentially dangerous capability for circumventing our safeguards,” Anthropic recounted in its safety card.
📖👀
Yes, it did.
ChatGPT-2 is too dangerous in 2019.
The lack of creativity in this marketing is disappointing…
They didn’t entirely miss the mark there. They publicly released the version after that and the world became worse. That certainly fits for some definition of ‘dangerous’, even tho it’s probably not how they were thinking.
Ya, they were pretty spot on IMO.
Hah I actually remembered this too, and people were still hyping Elon Musk at the time as well.
TBF the researchers knew what they had could be scaled into something gamebreaking which is how we got ChatGPT-3, but OpenAI made it sound like they already had it nailed down several years before it actually blew up. I think their unreleased examples they gave were a newspaper and short story written by AI which they said was indistinguishable from human material.
Let me guess, this super ai lives in Canada and we can never meet it, but it’s totally real.
You at give me another billion for data centers bro and you can meet it I swear bro just one more data center.
We do have a shitty ai data center up here, only about as super as a supermarket tho.
So there is a joke in the USA that if you don’t have a girlfriend you pretend you have one. She’s always super pretty, but your friends can never meet her because she lives in Canada.
I’m now curious to know if this joke was around before Avenue Q or not.
Edit: sounds like a yes!
Well, this caused me to learn something today. One of my favorite musicals is Avenue Q, which has an entire song about a girlfriend who supposedly lives in Canada. And I keep seeing this reference - but I keep thinking there is NO WAY that THIS many people know about Avenue Q (which is a pity).
And sure enough, TIL that this trope dates back to at least the 70s and is references in multiple TV shows and movies and such.
So Avenue Q was using an existing thing. Ah, well.
At least I know not to make Avenue Q references since there’s little chance they’ll be gotten. lol
Thats funny because here in Canada I knew a guy in highschool who had a clearly fake girlfriend who he said lived down south in the US
Nakes sense, though - you want the pretend person to be somewhere reasonable but not TOO close. lol
To be honest my fake girls friend was just a girl I had a crush on how lived a few cities away. I got busted lmao
awww, well I hope things worked out for you okay in the end. :)
For sure. It was embarrassing at the time but I was young
Ah, there we go - answered it for me, too! Thank you.
What? Do you think
AI company claims…
Isn‘t convincing? What gave it away? /s
I’m pretty sure Scam Altman tried this line some time ago for one of his supposed models.
GPT2
But can it start a timer
Man, I’ll start telling that to my boss whenever I miss a deadline. “Sorry boss, the code I made is too powerful, we can’t release it”
Like my dick
Does “it broke containment” mean it didn’t have permissions to anything and still managed to delete all the files it could find?
Anthropic lies almost about everything too … weird
crazy that the AI companies big selling point is always “our new model is TOO POWERFUL, it’s gone rampant and learned at a geometric rate, it enslaved six interns in the punishment sphere and subjected them to a trillion subjective years of torment. please invest, buy our stock”
Oh, funny, I also have sentient AI at home that I developed, but choose not to release it. My mom also created one accidentally while baking a cake but it was to powerful and she also decided to best destroy it like it never existed. You know, for everyones safety.
next time you or your mom have a cake you wish disappeared without a trace call me. I’m a… AI researcher
Grifters gonna grift.
No, its not too powerful. Its too chaotic. You cant control it.
EDIT: It seems I have misunderstood. I thought containment here referred to the harness, but they meant VM type of containment. I am still quite skeptical, but it looks like this model is quite good at finding and utilizing security flaws in software.
It may have blurted out something like “hey I know exactly how to end this economic suffering and all diseases globaly ! Its easy you just need to…”
Quick Hit the Red Button!!! Shut it OFF!!!
Bullshit
“Our AI has cost more money that it would take to solve world hunger, tanked the microchip economy, and ruined the lives of thousands of people we’ve had to let go… And it’s stupid as all fucking hell. What do we do?”
“Say it broke containment and it’s too powerful to release. Foolproof!”
Marketing
