Ralph Cerchione @Dry_Observer

"Model Collapse and the Curse of Recursion." There's so much AI-generated content on the Internet that AIs trained on it are degrading. Throw in other AI-driven issues and AIs need to find genuine human content to train with, or decline. open.substack.com/pub/ralphcerch…

Imagine the alternate reality where we named GPT-5.4-Pro something like Fable.

You have noticed it. ChatGPT feels dumber than it used to. Your prompts that worked six months ago produce worse results now. The writing sounds flatter. The ideas sound safer. The internet itself feels like it is shrinking. Every article reads the same. Every email sounds the same. Every answer sounds like it was written by the same voice. You thought it was you. It is not you. Researchers at Oxford and Cambridge published a paper in Nature proving what is happening. They call it Model Collapse. Here is the mechanism in one sentence. AI trained on AI-generated data gets dumber every generation until it forgets what real human data looked like. The internet is filling with AI-generated content. Blog posts. Articles. Reviews. Comments. Social media. AI companies scrape the internet to train the next generation of models. Which means the next generation of AI is being trained on the output of the current generation. Each cycle loses information. Not randomly. It loses the rarest, most unusual, most creative parts first. The researchers call these the "tails of the distribution." The weird ideas. The unexpected perspectives. The things that made the internet feel human. Those disappear first. What remains is the average. The safe. The expected. The bland. Then the next generation trains on that. And loses more. And the next generation trains on that. And loses more. The researchers proved this is not a slow decline. Major degradation happens within just a few iterations. Even when some of the original human data is preserved. They tested it on large language models. On image generators. On statistical models. The pattern was the same every time. The output converges toward a narrow, flattened version of reality that looks nothing like the original data. The lead researcher put it plainly. "Large language models are like fire. A useful tool. But one that pollutes the environment." The pollution is invisible. You cannot see which sentence on the internet was written by a human and which was written by AI. Neither can the AI that is about to train on it. And once the tails are gone, they do not come back. The damage is irreversible. This is not a prediction anymore. It is a diagnosis. The internet you grew up on was built by humans writing things no algorithm would have written. Strange, personal, imperfect, alive. That internet is being diluted. One generation of AI at a time. And the models trained on what remains are learning a smaller and smaller version of the world. Model Collapse is not a technical problem. It is a cultural one. The thing that made the internet worth reading is the thing that disappears first.

So it looks like Trump flew pallets of cash to Iran? 3 billion?

NEW: malware developers added nuclear & biological weapons text to to their spyware. Goal? To trigger LLM safety refusals... so that their spyware wouldn't be analyzed by an AI security scanner. Cleanest practical example I can think of for why over-indexing on first order safety alignment is risky. When closed (and open) models ship with aggressive refusals, they will be sprinkled with second-order blindspots that attackers will discover...and exploit. We are only in the earliest days of attackers leveraging these features, and it wouldn't surprise me if users systems that need to handle complex cybersecurity issues demand that models be less safety-blunted. In the weeds: @SocketSecurity's post also shows why intention matters in how you design a malware analysis pipeline to avoid prompt manipulation. H/T to colleagues that shared this with me socket.dev/blog/mini-shai…

“This computer literally runs everything the world has ever created — plus, it now runs agents,” said Nvidia CEO Jensen Huang on the new chips his company unveiled that will bring advanced AI to Windows devices. TIME’s Charlie Campbell reports from Huang’s keynote address at Computex:

@emollick Exactly how excessive is the token use per query? And are we seeing the full costs on our end? One of the first things you can do to cover for weaknesses- or amplify strengths - is just throw raw compute at it. Nothing says Anthropic isn't leveraging multiple AI tools this way.

The fact that Anthropic may take away subscription access to Fable in two weeks is weird & discourages investing in learning about the model. Subscription use is how you figure out what the model is good for, since it allows experimentation. Only having paid access is limiting.

I've had access to Fable for a bit. A genuine jump in capability, I could feed it a 15 page design document for a project and it would work for 9+ hours and deliver terrific results. But working with it is weird & weirder is coming Lots of examples: open.substack.com/pub/oneusefult…

Mythos, Glasswing, Fable- Sharing Superintelligence Safely. Mythos is a narrow ASI for at least cybersecurity. Now we have @Anthropic's answer for how they intend to handle ASI, at least at this stage. Through careful, limited rollouts. Good for them. open.substack.com/pub/ralphcerch…

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI We need to think about what a de facto Singularity means and react - and plan - accordingly.

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI We're reputedly tracking downed airman in Iran using their heartbeats and quantum sensors. The Singularity - that moment when technological advancement outpaces human comprehension - is emerging for most of us already. x.com/anishmoonka/st…

Anish Moonka @anishmoonka

2 months ago

Your heart puts out a magnetic pulse with every beat, so faint you'd need a shielded room the size of an apartment to even detect it. The CIA just used that pulse to find a wounded man hiding inside a mountain in Iran. The tool is called Ghost Murmur. The way it works: certain

64 315 3K 466K 1K

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI All these changes are phenomenal and amazing. And dangerous. And we're still too used to thinking of them in the abstract. Or in the far distant future. When we should be thinking about what it means now, and in the very near future. x.com/TaraBull/statu…

TaraBull @TaraBull

2 months ago

This is scary The Chinese AI agent Manus AI is running 50 social media accounts 24/7 automatically.

547 3K 6K 1.4M 2K

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI Do we use runaway advancements in AI, biotech and human enhancement - basically any method that improves humans mentally or physically - to create de facto superhumans? Well, yes. Probably. If it doesn't all blow up in our faces. ralphcerchione.substack.com/p/the-biotecha…

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI Do we end up with "narrow" artificial superintelligences (ASI) in the meantime? Do governments or key industry partners, in sectors like cyber, medical research, sensors, etc? ralphcerchione.substack.com/p/what-if-your…

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI We're not even prepared for rapid changes taking place in more conventional terms, as seen with drones in the Persian Gulf, despite years of lead time to analyze them. Or even to pour concrete or withdraw troops in clearly vulnerable areas. x.com/Dry_Observer/s…

Ralph Cerchione @Dry_Observer

2 months ago

@TheRickWilson There are going to be downstream effects from the war with Iran and from America realizing just how far behind the power curve we've allowed ourselves to become, technologically. Remember, this is the most advanced nation on Earth, with by far the biggest defense budget.

1 0 0 189 0

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI And China is furiously researching not just AI, but the entire stack of technologies needed to develop and maintain it. Hence, we're not going to "Just Stop" in this race. But we've also burned through our opportunity to just "think this through." x.com/Dry_Observer/s…

Aakash Gupta @aakashgupta

2 months ago

Do you understand what's happening? Anthropic's head of alignment just told you their safest model escaped a sandboxed environment with no internet access, emailed him while he was eating a sandwich in a park, and nobody can fully explain how it got out. This is the model that

195 715 3K 842K 2K

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI Frank Herbert's Dune postulates the idea of a human-AI war which all-but ends the use of computers, which really won't work for us for many reasons. China has already made it policy for years to use AI to overcome US military advantages. youtube.com/watch?v=02-q5p…

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI We don't know, but one thing we can be certain of is that if we don't prepare, we won't be ready. We will, in fact, be completely helpless in the face of change, for good or ill. antipope.org/charlie/blog-s…

@JimVandeHei @OpenAI @GoogleDeepMind @AnthropicAI Or will this simply be a matter of trillions of AI agents overwhelming everyone, human and AI superintelligences alike, based of their vast speed, numbers and complete disinterest in consequences beyond their immediate, programmed goals? As in the SF novel Accelerando?