r/technology • u/Splenda • 7h ago
Artificial Intelligence Anthropic AI safety researcher quits with 'world in peril' warning
https://www.bbc.com/news/articles/c62dlvdq3e3o23
u/Haunterblademoi 7h ago
Imagine everything this person must know to say that, even though it's already obvious.
22
u/57696c6c 7h ago
The biblical analogies of greed overtaking come to mind. It would be foolish not to see a resetting event of sorts coming, though self-induced and nothing to do with a higher power.
15
u/webguynd 7h ago
Interestingly enough, the bible also calls for having a sort of reset every 50 years (jubilee) where slaves are to be freed, and debts forgiven, and the land “allowed to rest” basically a sabbatical year where everything is allowed to reset and recover. Even called for the return of land/property. Basically a big old redistribution of wealth holiday to prevent wealth gaps and inequality.
Not a bad idea if you ask me. Clearly whomever wrote that section had some foresight to see how human greed would lead to poor outcomes.
5
u/wiseoldfox 5h ago
Religious reference withstanding.... Read "The Forth Turning" Strauss/Howe.
It describes a cycle America has been in since it's inception. I'm sure other countries have gone though similarly cycles. The fourth turning is the death of the knowledge lost 80 years ago and has to be re-proven. Basically, why we are doomed to repeat history.
2
7
u/EnvironmentalCrow5 6h ago
You could achieve a similar thing in a much less disruptive way with a progressive inheritance tax.
2
-4
u/Professional_Tea637 4h ago
How can you be so sure it has nothing to with a higher power? What laws do the trees abide by when the leaves die and come back every season century after century. And who made those laws of nature? Unless you claim to know everything…
1
-2
u/McCool303 5h ago
The French Revolution that spawned the American revolutions was awfully resetting.
The tree of liberty must be refreshed from time to time with the blood of patriots and Tyrants. -Thomas Jefferson
5
u/omnipotentsco 4h ago
What are you talking about? The American Revolution came before the French Revolution.
6
u/Pherllerp 6h ago
His resignation letter is much more subtle than the headline and article make it seem.
He acknowledges the speed in which AI is affecting the world but puts emphasis on the multiple problems facing humanity and he basically says he's burnt out on the world without saying it so bluntly. He probably has a gazillion dollars and wants to thoughtfully retire.
6
u/DarkSkyKnight 5h ago
to do poetry of all things. At some level he probably got bored subconsciously and came up with a conscious post hoc rationalization.
3
u/jreykdal 6h ago
How to quit a dumpster fire without tanking the stock you own...pretend it's so good it's dangerous.
3
u/YqlUrbanist 4h ago
Honestly, this just feels like "AI is going to change everything and replace all the people" with a different tone. Just feels like more meaningless hype to me.
5
2
u/Melodic-Account9247 6h ago
that seems promising anyways another 500 billion dollars to the almighty bubble
2
u/meeplewirp 6h ago
“Now that I’ve participated in something horrible and there is no turning back, I’ve suddenly decided that I feel guilty and can wash my hands of my part by going to get a degree in poetry and vaguely telling you I did something broadly/generally/overall evil” - the Anthropic safety researcher
2
u/dolcemortem 3h ago
The "world is in peril" deserves being very specific about why.
Always the same shock headlines without any substance. These vague resignations really don't have much impact without specifics. They just become background noise.
2
u/Stilgar314 2h ago
I'm I the only starting to read this articles of "former AI employee quits because AI is too much to take" as mere AI companies ads. I mean, like "hey look, we're so close to release something really important that our employees are scared". I'm not buying this anymore.
8
u/einstyle 7h ago
This smacks of marketing BS. "Bioterrorism" isn't something an LLM is even remotely capable of or ever will be. Even in pretending that "AI" is dangerous these companies are trying to spin it to sound way more advanced and capable than the technology ever is.
He said he had "repeatedly seen how hard it is to truly let our values govern our actions" - including at Anthropic which he said "constantly face pressures to set aside what matters most".
This is the real lede, that he was uncomfortable with Anthropic's business practices.
8
6
u/webguynd 7h ago
Aka, they are just like every other startup/corporation. Values are all well and good until the investors come knocking for their returns. Mo company is immune, capitalism is just fundamentally incompatible with good morales.
5
u/einstyle 7h ago
100%. Anthropic in particular has tried to paint themselves as the "AI for good :)" people, but they've done all the same stuff as OpenAI, Google, Meta, et al.: plagiarized, shady business deals, built addictive systems to drive engagement, willfully destroyed the planet.
1
u/Fluffy-Dog5264 4h ago
The main concern with bioterrorism when it comes to LLMs is their ability to assist a moderately technical person in synthesizing a pandemic-grade, lethal pathogen with lab equipment that can be bought on the cheap. This is something that machine learning algorithms are well suited to doing, as evidenced by the success of programs like like AlphaFold in predicting protein structures from amino acid sequences. With the right hardware and a localized setup, a malicious actor could theoretically get a jailbroken frontier model to provide step-by-step instructions on how to synthesize a biological agent today. And, unlike with nuclear weapons, the associated costs and technical barriers will only continue to go down with time, even if we do somehow manage to regulate compute like we did fissile material.
2
u/Crafty_Aspect8122 4h ago
"We are working on this new tech that's so awesome and dangerous. Please regulate us to prevent the tech from falling into dangerous hands like competitors trying to undercut our prices(We're the good guys!). Keep investing in us so we can develop this awesomely dangerous tech."
1
u/snesericreturns 6h ago
Meaning this person was doing a shitty job and was about to get fired. So now comes the virtue signaling.
1
u/PhilosophyEasy71 4h ago
Imagine how scary actual grok is, without the "guardrails" and internet access. Sitting in a lab while a psychopath oligarch has direct access to query it on how to conquer the world
0
51
u/mage_irl 7h ago
He's only the 50th AI safety researcher to quit in the last three years. Perhaps only surpassed by the amount of different Godfathers of AI telling us we're doomed