At least in my work, this is sort of like asking "If you don't enjoy CI/CD or the cloud, why not do without it?" It's becoming integrated into every process at this point.
And many employers now require you to code faster, which is only possible with AI tooling. They don't understand that coding faster isn't always better.
Is this related to what business your employer is in? In other words, is their business producing code, or is code written to support some other business?
Everywhere. Normally adding an extra endpoint to the REST API would take a sprint. Now PM expects you to do two. Only way to get that done is a lot of vibe-coding and delivering sub-par results.
I am amazed that anyone ever needed a "sprint" to "add an endpoint".
It usually speaks to the approach you are taking, but I also believe sprint-based planning leads development teams to make work fit into it instead of needing natural time to do it.
Let's remember that all of have probably built full systems over a weekend. What organizational dysfunction has led to us needing a sprint is what needs fixing.
Of course they do. Generally the end result from this is bad - reduced in scope, buggy, and/or unstable. But that's an issue with communication/expectations/management. It'll keep happening either because of politics or the fact that a sub-par deliverable still meets the needs of the overall organization. Until it doesn't.
My complaint with anthropic is actually the opposite. They seem too focused on building this suite of products (because they want lock-in), but they can’t even get the availability and speed on their models in an acceptable state. It really does seem like they’re falling behind google and OpenAI at the moment.
In 2024, a Chevy dealership deployed an AI chatbot that confidently agreed to sell a customer a 2024 Chevy Tahoe for $1. It executed a catastrophic business failure simply because it didn't know the logic was wrong.
Sure, you can patch that specific case with guardrails, but how many unpredictable edge cases are you going to cover? It only takes a user with a bit of ingenuity to circumvent them. There are already several examples of AI agents getting stuck in infinite loops, burning through massive API bills while achieving absolutely nothing.
You can contain a system failure, but you cannot contain a logic failure if the system doesn't know the logic is wrong.
You literally can contain a logic failure. If I execute logic on my computer that’s not connected to the internet it can’t get out of the box. Done. Contained
This is intentionally written vaguely. How do you limit that these implementations ensure Program() runs and does the right thing when there is no guarantee Math() or its components are correct?
Normally you could use a typed programming language, unit tests, etc, but if LLM is the ultimate abstraction programs will be written line above. At some point traditional software engineering principles will need to apply.
> If we assume that we fix bugs faster than we introduce new ones and we assume that the AI tools can improve further, the question is then more how much more they can improve and for how long that improvement can go on.
Why would we assume this? Historically it's obviously inaccurate since the world began with 0 software bugs.
That's not what I said. I said system design is not the exclusive domain of humans, so anyone thinking they possess some special knowledge of system design that an LLM isn't capable of obtaining are fooling themselves.
reply