Recently, we all experienced a Richter 8 earthquake in the fundamentals of the web in one way or another, due to PHP2Shell. Whether you were an end-user who couldn’t access the government’s website anymore, or you were a developer tasked with saving your company, we all felt it. And I won’t go into detail on this exploit! Engineers around the globe should all have the same shared gut feeling on the root cause of the issue: LLM-powered exploitation; CVE scanning.
Just take a look at this graph for me, it plots created CVEs over time:
Yes, you can argue whether “CVEs created” is actually a good statistic to measure the rising vulnerability of general software, but that’s not actually my point. My point is that more and more - even if “merely” potential - are found and reported. Clearly people are busier and busier trying to find these exploits, or even unintentionally! Any good coding agent mentions vulnerabilities in real time as it’s going through the codebase.
Try typing “Claude, find exploits, make no mistakes” in your project every once in a while.
Some of you would jest that this is utterly ridiculous, and let’s not pretend it isn’t (we live in an absurd time), but the funny/sad thing is that it works! And not just on your company codebase either. Did you know that you likely depend on thousands of codebases you have absolutely zero control over – if even you’ve the knowledge they exist in the first place? It’s called open-source. It’s the most beautiful thing humanity has ever come up with and it’s an absolute hellscape once you fathom it. If you’ve ever rented a raw VM in the cloud and seen the access logs completely spammed with crawlers the second you turn it on, it’s even worse with open source. Publish any library to any ecosystem and it gets hundreds of pulls per day just by automated systems. Any script kiddie can download an open source package and run it by a pay-by-the-token hypercapable all-knowing cybersecurity expert. Open source is largely run by volunteers or underpaid developers that still have a glimmer of hope and duty to humanity sparkling in their eyes. But that sparkle has been fading, rapidly, as LLMs are flooding these codebases with PRs, some malicious, some with good intentions, most of them not properly thought about. How are these kind maintainers supposed to review all of this while keeping their free little tool stable enough?
Enter PostgreSQL 19, the actual case study this blog post is about. We will learn from them.
PostgreSQL (which I will just call “Postgres” from now on) is the world’s favourite database and has a release once a year, around September or October. As of writing, it is the very end of October and we haven’t seen this year’s release, which is supposed to be version 19. It is an open-source project, which means anyone can see or modify the application for themselves. Every release cycle, Postgres allows users to propose changes and get their code merged into the product. Usually this stage takes around 9 months, then the remaining months are spent testing and patching until a release comes out, performed with the community at large.
And remember, everything you know touches Postgres: Your Instagram pictures, your Reddit comment history, all your ChatGPT chats, everything you ever did with a website running on Supabase. This product needs unbreakable, rock-solid review, test and release procedures, which is why they take so long.
Why is it delayed, then? In large part because LLMs keep finding issues. Up to ten significant features set for version 19 have been reverted, including the headline SQL/PGQ! Patches for bugs - or even for previous bug patches - have issues found!
It’s fascinating and horrifying to read the mailing lists for the current cycle: PostgreSQL: scary patch contest. (And do read this!) To quote the maintainers:
“I think it's pretty clear that none of these were as robust at commit time as we would like”
“It feels we are in an unprecedented situation here.”
“we are in a situation where we might be reverting multiple big features, and we don't even know if their removal will effect other changes”
Of course, this is healthy discussion and things you want to see from the people maintaining such a crucial product. It is better to admit you’re unsure of the impact than make a rash decision. It is good to acknowledge the situation and work through it communally, and make the hard decisions with the team, informed and thought through, rather than by each individual developer, each with unjust overconfidence and the fear someone may one day realise how incapable they actually are with dealing with the situation at hand by themselves.
So they delayed the release until they are happy.
It’s easy to judge the situation from outside and quote Miyamoto here and say “a delayed game is eventually good, but a rushed game is forever bad”, good job guys! But just like he never actually said this, it is never actually applicable in your role. You exist on the inside of this business; you are the actor, not the judge. You are being judged constantly, by many judges each with their own sense of justice and none of them agree, just like the Postgres maintainers being judged by the whole software world! We learn from them that you should not be afraid. Afraid to say no. To delay. To opine thy worries in the town square.
You should not be afraid to act. Delay it! Beg for a week of security testing! Beg to set up CVE scanning pipelines! Run a shitty audit with Claude, rather than nothing at all!
This touches the very foundation of what it means to be an engineer. Your job is not to write code or review architecture proposals or tell sales what is and isn’t possible, it is to fathom the chaotic landscape of software development holistically and make decisions nobody else can really ever understand unless they are also suffering beside you. The best judge in this matter is ultimately yourself. As someone who has a product, a codebase to maintain, who’s been put in charge of making decisions for a piece of software whether for client use or public use, you need to take the LLM threat very, very seriously.
As for the title question, “What will you do once AI’s ability to find bugs outstrips your pace of fixing them?” The answer is hereby clarified: you do your best and think holistically. That is what it means to be an engineer. You ask for more manpower. You set up defense-in-depth. You add risk clauses to the EULA. You document rejections of your objections. You hire pentesters. You triage features. You ask for delays.
LLMs are awesome and frightful tools.
We learn from the community as a whole.
Stay honest, and keep being an engineer.
28-10-2026
by Roy Prins