There is a blog post on sockpuppet.org arguing that we are not prepared for the upcoming flood of high-quality, LLM-generated vulnerability reports and exploits.

Now consider the poor open source developers who, for the last 18 months, have complained about a torrent of slop vulnerability reports. I'd had mixed sympathies, but the complaints were at least empirically correct. That could change real fast. The new models find real stuff. Forget the slop; will projects be able to keep up with a steady feed of verified, reproducible, reliably-exploitable sev:hi vulnerabilities? That's what's coming down the pipe.

Everything is up in the air. The industry is sold on memory-safe software, but the shift is slow going. We've bought time with sandboxing and attack surface restriction. How well will these countermeasures hold up? A 4 layer system of sandboxes, kernels, hypervisors, and IPC schemes are, to an agent, an iterated version of the same problem. Agents will generate full-chain exploits, and they will do so soon.

Meanwhile, no defense looks flimsier now than closed source code. Reversing was already mostly a speed-bump even for entry-level teams, who lift binaries into IR or decompile them all the way back to source. Agents can do this too, but they can also reason directly from assembly. If you want a problem better suited to LLMs than bug hunting, program translation is a good place to start.