Cover of Was I wrong about open models & cyber risks?

Was I wrong about open models & cyber risks?

Was I wrong about open models & cyber risks?

Listen for free

View show details

Season's Savings | 0.99 €/month for 3 months

5.99 €/month thereafter - terms apply. Cancel monthly.
Summary
 Last week coined the term “Black October“ after an onslaught of critical vulnerability disclosures, including PHP2Shell and a long-standing GitLab bug, signified the moment when the cyber risks of AI models started to materialize. It seemed like the news cycle had stabilized a bit when I started writing this, but then there was news about another major crypto security incident at Binance. The crypto industry has a very long history of hacks, so it’s not the best industry to indicate vulnerabilities translating into new societal harm. Still, we are very much still living through the implications of this transition.The best guess for what is happening in the cybersecurity space is that the behind the scenes work has only accelerated, bringing more people and resources in to try and patch the internet — the Wall Street Journal reported that cyber first responders are strained, which seems like the system working. We’re living through this plot of an exponential rise in vulnerabilities, which will take a long time to level off at the top of the sigmoid.A few weeks ago, before the rapid acceleration in vulnerability disclosures and the uptick in reported hacks, I wrote the following take. It’s been getting a pretty consistent barrage of feedback, some hopeless dunk attempts and some reasonable discussion, so I wanted to unpack where I stand.Open-weight Mythos in April would’ve been fine?When Claude Mythos was announced, it was previewed as if it was a new class of cyber weapon, where if it ended up in the wrong hands it would’ve caused mass societal destabilization.It seems like this was wrong.By all measures, GLM-5.3 is the model that crosses that threshold of capability, and there’s little public evidence that much has changed, and we’re over a month out of the model weight release.In fact, I would go further. If Claude Mythos was accidentally released as open-weight, it seems like the world would have been more or less fine. Yes, there would be a clear increase in cybersecurity incidents and it would be bad for society if the model leaked, but it would be a scenario that looks more like an acceleration than a step change in risk.The leading proponents of this new wave of open-weight fear mongering have effectively been making a falsifiable prediction — that the current open-weight models are going to cause substantial, not seen before AI harms by crippling our cyber infrastructure.Based on all the evidence I have, which I’ve unpacked above, they’re set up to be wrong. At least now, after so many years of debating open weight risks, we will start to get some real answers.To start the reflection, I wanted to do more due diligence and track down some receipts on the claim that the AI safety community was reacting as if open-weight Mythos would be an actual disaster, and not just a painful headache. This was distinctly my impression having lived through the online discourse. I’ve gotten criticism that the frontier AI community never said it would be a “true disaster,” but regardless of exactly what they said, it is always about how people hear and internalize your information.Here are some examples of the sort of language that I felt was too exaggerated and from prominent people, which spilled into the public and more importantly federal government perspective that this was a cyber weapon include:* Rob Wiblin, host of the 80,000 hours podcast, on his own podcast (April 10, 2026): “If it fell into the hands of criminals or hostile state cyber actors, it would be an actual disaster.”* Anthropic, in the Project Glasswing announcement blog (April 7, 2026): “Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely. The fallout—for economies, public safety, and national security—could be severe.”For one that is actually more reasonable, to illustrate the fine lines between correct and exaggerated, from Zvi Mowshowitz (who I don’t agree with a ton of things on) on April 9, 2026: “Claude Mythos would give attackers a cornucopia of zero-day exploits for essentially all the software on Earth … It would be chaos.” To date, this chaos has been confined to those running large-scale digital products, which is good, but could change. I largely still stand by my above writing a few weeks ago, but I’m nervous about it. I’m not a cyber expert, I’m working hard to make the most educated predictions I can. It’s a fine line between “all of our tech companies need to work extra hard to keep us secure” and “public infrastructure is actually going down.” It is very clear as of writing this that there are way more critical vulnerabilities which are being found in popular tools and fixes are being built.For example, Epoch AI recently published a nice post on their internal work using GLM-5.3 to try and find vulnerabilities and chain them into what could be an attack. ...
adbl_web_anon_alc_button_suppression_t1
No reviews yet