So far the studies point to study authors having a profound misunderstanding of what’s happening. Which isn’t surprising, since any study right now requires speculating about what’s important and impactful in a new and fast-moving field. Very few people are good at that, and most of the ones who are are not running studies.
Until the LLM is wrong and Bob passes the erroneous result off as accurate, reliable and vetted by a knowledgeable person. At that point Bob is not producing a useful result. Then it becomes a trap other people might get caught in, wasting valuable time and energy.
It's a spectrum and we don't have clear notches on the ruler letting us know when we're confidently steering the model and when we've wandered into vibe coding. For me, this position is easy to take when I am feeling well and am not feeling pressured to produce in a fixed (and likely short) time frame.
It also doesn't help that Claude ends every recommendation with "Would you like me to go ahead and do that for you?" Eventually people get tired and it's all to easy to just nod and say "yes".
I have to disagree that Bob will be a better producer, although I do agree that Bob will produce more. In this scenario, Bob isn't clear on which LLM output is valid and important and which is erroneous and misleading; I think that's a pretty critical distinction. It's the kind of thing that might go undetected for a long time, until a particular paper turns out to be important and it's discovered that it's also entirely wrong, wasting a lot of time and energy.
Sounds like you're still thinking of Bob as a researcher.
In production, there would be no "paper"; just some software/hardware product.
If there was a problem, that would be fairly obvious, with testing (we are going to be testing our products, right?).
I have been wrestling all morning, with an LLM. It keeps suggesting stuff that doesn't work, and I need to keep resetting the context.
I am often able to go in, and see what the issue is, but that's almost worthless. The most productive thing that I can do, is tell the LLM what is the problem, on the output end, and ask it to review and fix. I can highlight possible causes, but it often finds corner cases that I miss. I have to be careful not to be too dictatorial.
It's frustrating, as the LLM is like a junior programmer, but I can make suggestions that radically improve the result, and the total time is reduced drastically. I have gotten done, in about two hours, what might have taken all day.
I think the difference between the workflow you describe and the description of Bob's is that you have a pretty good idea of what a working solution would look like. In my reading if the article Bob does not.
In my opinion, the software developer analogue of Bob would be someone who would often reach for the LLM as it's nearby and easy. Maybe at first they would be careful about reading and vetting the model's output. Over time they might grow comfortable and overly confident with the model's output and pay less and less attention. As they take on more complex tasks they begin to understand less and less about the LLM tooling output but they don't really notice, it all looks good and tests are passing. Eventually we see a production problem, maybe even an outage. When we narrow down the issue to a PR with Bob's name on it and ask him how it led to the production issue, Bob tries to be helpful but struggles to understand his own PR.
Yeah, I'm very much a "production" person, so that's my lens.
For me, I started off with Machine Code, but these days, I have no idea what's going on, below the compiler. I know enough to be dangerous, looking at a stack dump, but that's about it. My days of hex keypads are long gone.
I still get stuff done, that I could only dream of, back then. LLMs are really just another step on the evolutionary ladder.
I'm old enough to remember when calculators were banned in the classroom, for just about exactly the same reasons that people are complaining about LLMs.
Anyway, the die is cast. We'll have to see how things turn out.
I think we already know what we need to do: encourage people to do the work themselves, discourage beginners from immediately asking an LLM for help and re-introducing some kind of oral exam. As the article mentions, banning LLMs is impractical and what we really need are people who can tell when the LLM is confidently wrong; not people who don't know how to work with an LLM.
I hope it will encourage people to think more about what they get out of the work, what doing the work does for them; I think that's a good thing.
I think we'll get there. We need to get at least some AI bust going first though. It's impossible to talk sense into people who think AI is about to completely replace engineers, or even those who think that, while it might not replace engineers, it's going to be doing 100% of all coding within a year. Or even that it can do 100% of coding right now.
There's a couple unfortunate truths going on all at the same time:
- People with money are trying to build the "perfect" business: SaaS without software eng headcount. 100% margin. 0 Capex. And finally near-0 opex and R&D cost. Or at least, they're trying to sell the idea of this to anyone who will buy. And unfortunately this is exactly what most investors want to hear, so they believe every word and throw money at it. This of course then extends to many other business and not just SaaS, but those have worse margins to start with so are less prone to the wildfire.
- People who used to code 15 years ago but don't now, see claude generating very plausible looking code. Given their job is now "C suite" or "director", they don't perceive any direct personal risk, so the smell test is passed and they're all on board, happily wreaking destruction along the way.
- People who are nominally software engineers but are bad at it are truly elevated 100x by claude. Unfortunately, if their starting point was close to 0, this isn't saying a lot. And if it was negative, it's now 100x as negative.
- People who are adjacent to software engineering, like PMs, especially if they dabble in coding on the side, suddenly also see they "can code" now.
Now of course, not all capital owners, CTOs, PMs, etc exhibit this. Probably not even most. But I can already name like 4 example per category above from people I know. And they're all impossible to explain any kind of nuance to right now. There's too many people and articles and blog posts telling them they're absolutely right.
We need some bust cycle. Then maybe we can have a productive discussion of how we can leverage LLMs (we'll stop calling it "AI"...) to still do the team sport known as software engineering.
Because there's real productivity gains to be had here. Unfortunately, they don't replace everyone with AGI or allow people who don't know coding or software engineering to build actual working software, and they don't involve just letting claude code stochastically generate a startup for you.
> Or even that [AI] can do 100% of coding right now.
I don't actually think the article refutes this. But the AI needs to be in the hands of someone who can review the code (or astrophysics paper), notice and understand issues, and tell the AI what changes to make. Rinse, repeat. It's still probably faster than writing all the code yourself (but that doesn't mean you can fire all your engineers).
The question is, how do you become the person who can effectively review AI code without actually writing code without an AI? I'd argue you basically can't.
My boss decreed the other day that we’re all to start maximising our use of agents, and then set an accordingly ambitious deadline for the current project. I explained that being relatively early in my career I’ve been hesitant to use any kind of LLM so I can gain experience myself (to say nothing of other concerns), and yeah in his words I’ve “missed the opportunity”
Unfortunately in the majority of organizations, the idiots are at the wheels. It's not people with actual experience of how engineers do things, that dictates what those engineers should do.
Interesting, we only have generic 'use AI' in our goals. Though its generic framing probably indicates more company's belief in this tech than anything else.
> The question is, how do you become the person who can effectively review AI code without actually writing code without an AI? I'd argue you basically can't.
I agree, and I'd go a step further:
You can be the absolute best coder in the world, the fastest and most accurate code reviewer ever to live, and AI still produces bad code so much faster than you can review that it will become a liability eventually no matter what
There is no amount of "LLM in a loop" "use a swarm of agents" or any other current trickery that fixes this because eventually some human needs to read and understand the code. All of it.
Any attempt to avoid reading and understanding the code means you have absolutely left the realm of quality software, no exceptions
I'm not sure we can hold individuals responsible for signing these non-disparagement clauses. They often don't have a lawyer to review the paperwork and, I am sure, employers like Facebook aren't going to wait for a new hire to have a lawyer review that paperwork. There's a real pressure to sign everything with HR and get on to starting your new role.
In this case the author is an attorney, so can presumably understand a contract she signed.
And the non-disparagement wasnt in her hire agreement, it was in her severance agreement, in exchange for a negotiated amount of money. Author was wealthy enough to afford dedicated lawyers to review.
If you can't hold individuals responsible for signing contracts on the premise that they might not read them, you've simultaneously invalidated most contracts, increased the cost to enter a contract (essentially subsidizing the law industry), and severely limited the ability for individuals to enter into contracts — not to mention infantilizing adults who are fully capable of reading a document before signing it.
Furthermore, this was a severance agreement, not employment agreement, so having a lawyer review your contract is not going to delay you from starting your role.
I have my issues with this situation, but "they might not have read/understood the contract" isn't one of them.
Oh no, I wasn't clear: I am sure people read these agreements. I think most of the time, they don't understand them. Also, we can't always understand how these agreements might be applied even if we do think that we understand them.
I don't think we can hold people responsible for these types of contracts if they are under duress, for instance, under threat of the loss of their job. In this case the person signed in order to get the promised severance package, without which they wouldn't be able to continue with life-saving medical coverage, in my opinion that would also be under duress.
Love this! I got a USB C multimeter and used it to test the output of two dozen chargers. Wanted to see if they supplied the voltage that was advertised. Funny enough, AOHI was the only brand whose chargers actually increased their voltage as my current draw went up. It was like the engineers knew about the resistance in the wire and decided to compensate by upping the voltage slightly.
As an alternative, you could get a stand-alone USB-C power meter which can be used with any cable. That way, when the cable breaks, you don't have to buy a new power meter. Here is an example of one such product (though I've never used this model): https://www.amazon.com/Adapter-Voltage-Current-Extension-Con...
It's worth keeping an eye on this HP-rental-laptop thing.
Personally I think it will be a big headache for HP, people can be hard on laptops and HP is already not excited about consumer support (i.e. mandatory 15 minute wait time for support calls). But if they make it work, I think there's probably a good number of people who feel like they need a laptop but don't care so much about the specifics and want to keep their costs low (as all of their costs appear to be rising right now).
Rental seems to be about corporate laptops. Companies just want things to work at a predictable cost. They are already replacing laptops after 5 years even if they work. They are already replacing a few laptops that break in less than that 5 years. In short they are already renting the laptops, they are just paying the price upfront and then using accounting to balance it out. Rental just moves the accounting, but otherwise nothing changes.
For consumers who don't replace their laptops on a schedule it makes less sense.
I assume they are talking about the "This application was downloaded from the internet" warning, which I also don't like. Requiring dollars for signing and then _still_ showing a warning when someone installs your application seems crappy to me.
reply