Dangerous Open Models: When Safeguards Become Optional

The shutdown of Fable might be our first glimpse of an uncomfortable future.

Not because AI models are getting too powerful—but because safeguards might become completely meaningless.

The Guardrail Advantage

Here's what keeps me up at night: commercial models like Anthropic's Claude come with layers of guardrails built in. Try to get them to help with active exploits, and you'll hit a wall. Ask them to assist with certain bioengineering tasks, and they'll politely decline. These companies invest heavily in alignment and safety measures.

But Open Models Are a Different Story

But open models? That's a different story entirely.

Right now, some open models are already outperforming Opus. They're catching up fast—not just in capability, but in the specific knowledge domains that make them genuinely concerning. Models trained specifically on exploit databases. Models with deep bioengineering knowledge and no restrictions on how that knowledge gets used. Models optimized for things that most platforms explicitly prohibit.

And here's the thing: this cat isn't going back in the bag.

You Can't Recall a Model

We can debate whether these models pose existential risk to humanity. But if they do—if we're genuinely approaching that threshold—what exactly happens when those capabilities exist in open weights that anyone can download?

You can't recall an open model. You can't patch it remotely. You can't add guardrails after the fact.

Will we see bans on certain model architectures? Almost certainly. Will that stop determined actors from accessing them? History suggests otherwise.

A Dark Web of AI Models

I can easily envision an underground market emerging—a dark web of AI models. Not because people are inherently malicious, but because there will always be someone who believes the restrictions go too far, or that they have a legitimate use case, or that information should be free regardless of consequences.

Black hat hacking models. Bioengineering models without ethical constraints. Models trained on data that commercial providers would never touch. The lag between commercial and open models is shrinking every quarter.

Safeguards Only Work When You Control Distribution

The conventional wisdom says we need better safeguards. But safeguards only work when you control distribution. Once the weights are out there, once the model is truly open, safeguards become optional. And optional safeguards aren't really safeguards at all.

I don't have solutions here. After 25 years building software, I've learned that technology rarely moves in the direction we wish it would—it moves in the direction it can. And right now, every trend points toward more capable open models with fewer restrictions.

This Isn't Alarmism—It's Realism

This isn't about being alarmist. It's about being realistic.

We're building increasingly powerful tools and simultaneously making them increasingly accessible. The Fable shutdown might be the first domino, but it won't be the last. The question isn't whether we'll face this challenge—it's how we'll navigate it when open models reach parity with today's most capable commercial systems.

And based on current trajectories, that's not a distant future. That's 12-24 months away.

I'm curious how other builders and engineers are thinking about this. If you're working on AI systems, orchestrating agents, or just deep in the weeds of what these models can actually do, I'd be interested in your perspective on where this heads. Feel free to reach out—I'm always looking to learn from people who are actually building in this space.