TL;DR
- AI has not created the holes in software. It has sharply cut the cost, expertise and time needed to find them, and bad actors are exploiting applications built on vulnerable software.
- Building enterprise software is getting cheaper, but fixing legacy software will cost drastically more. Every newly exposed vulnerability becomes a patching job that runs up and down the stack and needs armies of engineers.
- Patching is a solid business, but governing agent actions is the bigger hill to climb. Agents will not be stopped, so their actions have to be observed, controlled and, if they go rogue, contained.
- Like America’s debt crisis, this is a pause, not a period, and security budgets will rise and settle over the next two years. Build your 2027 AI roadmap with observability at the center.
- Agents are not going to be stopped, so someone has to watch them. At Moative, we help mid-market companies build their 2027 AI roadmap and budget with observability at the center of it.
There is a lot of fear-mongering about how AI is making critical software exploitable. The fear-mongering is true. It is not that AI agents are attacking your bank account. It’s the applications built on vulnerable software that is being exploited by bad actors, that make software security a crisis.
All said, we cannot deny the transient problems that the software industry has to solve. The transient problems could be a billion dollar heist here or a manipulated election there. The attack surfaces are wide today.
The cost of building enterprise software is plummeting. But the cost to fix legacy software is going to increase drastically. It is not that vulnerabilities didn’t exist. The cost, expertise, and time to explore these vulnerabilities have fallen down dramatically.
You and I can team up to raid a small rural bank. But, seriously, keep me out of it, if you plan to.
If a bank has 100 vulnerabilities, exposing 200 more and asking them to fix them all is a good business. Agentic observability and security systems will make hay. Patching vulnerabilities is a solid business. But governing agent actions is the emerging hill to climb. Agents are not going to be stopped. Their actions have to be observed, controlled, and if they go rogue, contained. Checking every agent action used to mean calling a large model each time, and how a new kind of small model changes that math is the subject of AI Checking AI.
AI consulting companies will have an interesting time as well. As legacy systems get exposed and new vulnerabilities emerge, patching job goes up and down the stack and across the enterprise. These need armies of engineers.
Eventually, cyber models from frontier labs will make it hard to exploit agents in production. But the context of individual enterprises, their systems, and their agentic action surfaces will be better handled by independent agentic security companies that counter-act vulnerabilities created by frontier models. Permissions, authority and accountability stay outside any model, which is why each company has to design how its own agents act, the case we made in The Dawn of Enterprise Agency Engineering.
Ironically, this security crisis is like the looming debt crisis in America. It may pop. Or, who knows, we may print money or devalue gold or pump crypto and whatnot. I am long America. AI is going to make it a continuing superpower. So the debt crisis will be a pause, a comma, and not a period. The security debt crisis is the same. Collective action will fix it. Valuable agentic security companies will emerge.
In not too far a future from now, new systems will be attack resilient. Old systems will be patched up through AI and consulting efforts. Security budgets will raise and settle down over the next two years.
If you are building your AI roadmap and budgets for 2027, talk to us. We will help you refine it with observability at the center of it.
Frequently asked questions
What is the security debt crisis? The security debt crisis is the growing pile of vulnerabilities in legacy software that AI has made cheap to find and exploit. The vulnerabilities always existed. What has fallen dramatically is the cost, expertise and time it takes to explore them. Like America’s debt crisis, it is likely to be a pause rather than an ending, and collective action will fix it.
Is AI making software easier to hack?Yes, but not in the way most headlines suggest. The bigger threat is not AI agents attacking bank accounts directly. It is bad actors exploiting applications built on vulnerable software, now that finding those weaknesses takes far less cost, expertise and time.
Why will fixing legacy software cost more?Building new enterprise software is getting cheaper, but old systems carry vulnerabilities that are now easy to expose. A bank that knew about 100 vulnerabilities could face 200 more. Each one becomes a patching job that runs up and down the stack and across the enterprise, and that work needs armies of engineers.
Why do AI agents need observability and governance?Agents are not going to be stopped, so the question is how to manage what they do. Their actions have to be observed, controlled and, if they go rogue, contained. Patching vulnerabilities is a solid business, but governing agent actions is the emerging challenge.
Will frontier AI labs solve AI agent security?Cyber models from frontier labs will eventually make it hard to exploit agents in production. But every enterprise has its own context, systems and agentic action surfaces. Independent agentic security companies will be better placed to handle those, including vulnerabilities created by frontier models themselves.
How should companies plan AI security for 2027?Expect security budgets to rise and then settle down over the next two years. New systems will be built to be attack resilient, while old systems get patched through AI and consulting efforts. When building a 2027 AI roadmap and budget, put observability at the center of it.
