AI Devs, A Double-Edged Sword

You spend hours, days, weeks, months, and sometimes even years crafting intricate pieces of software. It's a labor of love. But also, sometimes, it's a labor of frustration. There's a lot of tedious, repetitive work involved. And then there are the bugs, the weird, gnarly problems that can take ages to track down and fix.

Now, imagine if you had an assistant. Not just any assistant, but a super-smart, hyper-efficient, always-on assistant that could take care of the tedious stuff for you, help you find bugs, and even suggest ways to improve your code.

That's the promise of AI in software development. The new generation of AI tools (like Github Copilot and, more recently StarCoder) use advanced language models like GPT-4, to help developers write code more efficiently and effectively.

Sounds great, right?

Well, yes. But as always, there's a catch. A big, complex, multi-layered catch that we need to unpack. While these AI tools can improve productivity, they can also create problems for your team.

First, let's talk about what these AI tools can't do. While they can reason about a few hundred, or even a few thousand lines of code, they can’t reason about the broader context of your system. This context lives across multiple files and repos. It includes code, configuration, software and infrastructure. It even includes the usage patterns of your system, hot-spots created by the unique properties of one or two particularly large customers. Powerful as these models are, they can’t reason about all this at once. They can't predict their actions' second and third-order effects or foresee how a small change might ripple through a complex system and cause unexpected problems.

This is a bit like if you asked a novice chess player to make a move in the middle of a game. They might be able to see a good move in the immediate vicinity, but they won't be able to predict how that move will affect the rest of the game.

While AI tools can speed up the coding process, they can also introduce new risks and challenges. They generate code that looks good on the surface but breaks in complex unforeseen ways when it hits production. The particularly troubling aspect of this is that teams will take longer to remedy these issues because they didn’t write the code in the first place. Much like autonomous driving systems, this problem will actually get worse as we trust these systems more and more. We will spend less and less time double-checking their work, so when this do go wrong… they will go very wrong.

I should clarify I’m not opposed to these tools being deployed, far from it. The speed at which teams will be able to innovate will be dramatically increased as long as we have corresponding innovation in how we deploy and operate this deluge of code. Otherwise, we risk negating productivity gains in development with productivity losses in operations.

So what does that look like?

Ultimately we need to fight fire with fire. A new generation of AI-powered operations tools will be needed to prevent, predict, manage, and resolve these issues. Much like with development workflows, these tools will augment humans, allowing them to operate more software at a lower cost.

At availabl, we believe that automated detection is the first layer of defense. If systems are evolving rapidly, a handful of traditional alarms aren’t going to cut it. The creation and maintenance of these alarms would become a bottleneck in itself. Instead, we continuously scan your cloud environment for issues, alerting you at the first sign of trouble.

We are also building tools that capture the broader context of your system via our Slack integration. Our GPT-powered bot learns from each incident your team resolves. This lets us share knowledge across the team, suggest avenues of investigation or even potential mitigations.

Advances like these, paired with improved CI/CD tooling, might just let us have our cake and eat it too. As with all disruptive technology, there will be an adjustment period while we get to grips with the newfound power of AI development, but in the long run, the future looks promising.