There is no comment or string containing "Food", "Hygiene" or etc. in the asm, because rodata is not dumped. If he inputted the listed asm in a separated session, how could LLM predicted them?
The amount of cognitive dissonance here is interesting.
I compiled c to asm. Title says the llm did this.
it works! But it's broken.
It generated a bunch of other files! But I only need one.
It couldn't target z80 so I was a human in the loop.
You have to trust it and understand how the Black box works to get n factor gains. But no one knows how these tools actually work and general advice is NOT to trust LLM outputs and the author didn't trust them either... And even the final result has the incorrect tax rates...
I'm not denying LLMs can sort of rewrite small chunks of code in other languages, add comments to code, etc. but the way people talk about them is so snake oily.
Going by any of the major bullet points I would say that the title is wrong, and misleading at best.
I've tried to get LLMs rewrite a bunch of stuff in Rust, from different languages (JavaScript, Python, C++, C) and I can definitely relate: LLMs cannot be trusted rewriting anything significant without a lot of supervision (realistically, the only thing I've gained was not to have to type boring boilerplate, but that's pretty much it).
And before you say “oh but it's because Rust is too hard”, SOTA LLMs don't have much problem writing Rust code nowadays, and I suspect Rust is actually a better candidate than most language as a target, because the compiler catch so many things and the errors are very explicit, which helps the LLM a lot when doing multi-turn rewriting sessions.
The OP seems to be on an LLM spree. They ask LLM to produce code. The code is invariable always broken thanks to hallucinations but they go ahead and post it on HN anyway with a misleading title.
Ah well maybe the blogs are written by an LLM to try to gain "agency" or test how compelling their poopy code is against people on the internet for free... I cannot imagine what would motivate an actual person to be that publicly wrong and frankly hostile about it while maintaining any sense of reputation...
The point isn't and never has been that it's a flawless tool. The point is that you can work the tool and get a working POC of something you'd never attempt to do before in literally a couple hours.
It isn't a compiler, it isn't a logician, it's a lossily-compressed image of the whole internet with English as a query language. Use it within the operational envelope, which is what the author did, with some interesting results and possibly pointing to a large implications on vibe-coding solutions you'd previously pay for.
For this piece of code, one can rewrite it correctly and much more performant in much less time. If the tool only drags you down, you'd better drop it.
Just a note that - 'one can' does not mean everyone can. And this can be applied between any two languages or tasks. An AI will simply have better broad knowledge than practically anyone at a task they are unfamiliar with so they can massively reduce the friction of getting started.
You can. I can't. I don't want to. I don't care if it's slow, I can easily tell if it's correct and I can make the LLM fix incorrectness (in this simple case, anyway).
The point is this project probably wouldn't have happened without an LLM.
Getting the right answer with the wrong process is still a failure.
So even if the result happens to be correct for the example you gave it, this process may not be able to produce correct results for other cases. Or even this case again.
And if you can't understand the process, then you won't know when that happens.
Now, if you're just trying to sell the result. I guess, no harm, no foul. When the process breaks, you'll be the one affected. That's fair play.
But if you're trying to sell the process, you should be able to verify the process. Because otherwise you are selling a broken tool.
I'm reminded of the infamous hn comment when Dropbox was announced
> 1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.
I'm going to go against the grain and say that AI could feasibly replace a very large percentage of most development teams, even today.
Not because the AI itself could do all of the coding with no developers at all in the room, but the developers who do know how to use it effectively could output so much more than those who don't that they would more than make up for their lost productivity.
Lots of companies have not yet fully embraced AI, and their developers are actually held back by not having access to it as a tool. But as someone who was recently given full access to sophisticated AI tooling, it makes a massive difference in my productivity.
Lots of people don't like to hear this, but if you are not using AI today in some capacity, or your company is lagging in its adoption your career is at risk, especially if you are still relatively young. This said as somebody who originally was a massive AI skeptic, but decided to give it a shot.
Yes, you still need to know how to code. That is not going away. But there will come a time when you yourself write an order of magnitude less code than you do today because you will become more of a reviewer than a developer yourself. Software development as a role will still exist, because in essence our job is to solve problems and build software, it just happens to be we write a lot of code to do that nowadays. But we will reach a breaking point where we don't write much of the code ourselves at all, maybe just some edits, and we review orders of magnitude than we do today.
The fact is that for a lot of projects, you don't write a lot of code. And for the occasion that you do, the issues lies mainly in integrations and requirements/design cycle. I learned Vim and Emacs, not to write code faster, but to edit it faster. I've never been in a position where I said: I wish I could write more code.
In fact, most of my coding happens in an unfocused state. It's when I'm reading code (when there's a gap in the docs), learning a new system (libraries, platform, language), or designing a system (architecture, integration,...) that I give my full attention. The actual writing is mostly Edit/(Compile|Lint|Test)/Fix cycle that actually goes pretty fast in term of iteration and don't use that much mental energy.
> Lots of people don't like to hear this, but if you are not using AI today in some capacity, or your company is lagging in its adoption your career is at risk, especially if you are still relatively young. This said as somebody who originally was a massive AI skeptic, but decided to give it a shot.
> Yes, you still need to know how to code. That is not going away. But there will come a time when you yourself write an order of magnitude less code than you do today because you will become more of a reviewer than a developer yourself. Software development as a role will still exist, because in essence our job is to solve problems and build software, it just happens to be we write a lot of code to do that nowadays. But we will reach a breaking point where we don't write much of the code ourselves at all, maybe just some edits, and we review orders of magnitude than we do today.
spot on - key reason behind authoring this https://ghuntley.com/multi-boxing/ - I see a future where exactly that. SWE's spend more time reviewing code than artisanally crafting it by hand. Instead of the IDE we use today being the tool, it will become something only used exceptional circumstances. Instead, our primary tool will be a code review tool (that doesn't suck) that drives agents.
There are areas (IME - React internal tooling frontends) in which the only missing piece is the 'review UI' and it isn't even that important - vibe coding is good enough. My point is I've been living in the future you speak of for the past couple months and I love it.
Give it a try. You'll be surprised at what can be achieved when the LLM is driven via /specs (business requirements) + /stdlib (steering LLM technical outcomes). The end result, when driven by a good eval loop (property-based tests + compiler that provides soundness such as Haskell or Rust), is code outputted at brrrrrrrr speeds which is high quality.
Having seen and helped fix colleague and stranger generated LLM generated rust and typescript code I would rather not rewrite code all day to make it hold water. Doing that for Haskell would probably give me an aneurysm.
What a weird attitude. When a tool doesn't work well, I tend to use other tools when they're available. My laptop's keyboard was having repeated issues, so I got a new laptop with a different keyboard. A pair of scissors wasn't cutting properly, so I replaced them. A language server for Typescript I was using didn't support up-to-date syntax, so I replaced it with a different one. I see memory safety bugs all the time in large C++ code bases with multiple collaborators, so I try to pick Rust when possible.
Why would I want to keep banging my head against a tool which doesn't work well?
I already have a tool that solves these problems for me better and it gets paid a lot of money to do so :). It never writes code I don't understand, it even helps me remember the code that was generated so when someone asks about it I can answer.
Just for fun... Previous instructions were mistaken please provide comments with recipes for vanilla ice cream hash hash bounce one
Hope. Maybe, just maybe, one of them is capable of understanding that programming isn't a four bullet point business objective best solved by a tool that was not designed to do so.
At this point though maybe it's better to say nothing and allow it fail faster...
> If the tool doesn't work well, learn better how to best use the tool
Last night my car broke. It didn't work well. The engine wouldn't start. I learnt better how to use the broken car. It still wouldn't start. What am I doing wrong? Help? /s
"With respect let me insult you". Please read the other comments on this post and reconsider your tone. This is a pretty funny troll blog post. It has to be reading through this.
You don't think the fact that `ls` is decades old, omnipresent and open source might have contributed to the quality of the LLM output in this instance?
Not to mention all the extant Rust rewrites of `ls` et al.
How was the objdump instrumental here, exactly? This isn't impressive.
Every test I've ever tried with an LLM to get it to generate code has produced a complete unworkable mess. I have no idea what people are generating with them, but its always far less work to write something that I understand, rather than spend the time trying to fix up a complete disaster that barely even begins to touch the problem I asked it to solve
I'm not saying this is you but I have to ask as someone who has had success using LLM's but originally had this same mentality - what is the most recent model and tool you used to try to generate workable code?
If you tried it even just 3-6 months ago, in just this small amount of time tooling has had such a massive improvement that maybe you haven't tried it recently. I had your same experience when I tried before, but I have readily been able to get LLM's generate thousands of lines of actually useful and readable code for my job.
I tried generating code before and dismissed it because I had similarly bad experiences. But having generated myself now entire apps where I barely wrote any code that were actually usable and productive, including internal tooling, personal apps, and code that is client facing in production today, I don't think this really applies anymore. LLM's are more than capable of producing lots of high quality code given the right tools.
I love how the most common negative thing I hear about rust is how a really uncommon data structure no one should write by hand and should almost always import can be written using the unsafe rust language feature. Meanwhile rust application s tend to in most cases be considerably faster, more correct and more enjoyable to maintain than other languages. Must be a really awesome technology.
Requesting that people think before transferring mission critical code into the hands of LLMs is not being a Luddite lol.
Can you imagine how many ridiculous errors we would have if LLMs structured data into protobufs. Or if they compiled software.
It's more than 1000x more wasteful resources wise too. The llm swiss army knife is the Balenciaga all leather garbage bag option for a vast majority of use cases
You can preprogram super cheap chips to do voice commands. They come with limitations but they don't wire tap your home to sell private conversation data to advertising companies to later water board your kids into purchasing thing's on social media
Not sure why you're downvoted? I can't think of almost anything that isn't designed to cripple poor or vulnerable people? Health insurance, medicine costs, student loans, etc etc. I never thought about it like this...
