The dichotomy between the people who are "orchestrating" agents to build software and the people experiencing this less than ideal outcomes from LLMs is fascinating.
I don't think LLM for coding productivity is all hype but I think for the people who "see the magic" there are many illusions here similar to those who fall prey to an MLM pitch.
You can see all the claims aren't necessarily unfounded, but the lack of guaranteed reproducibility leaves the door open for many caveats in favor of belief for the believer and cynicism for everybody else.
For the believers if it's not working for one person, it's a skill issue related to providing the best prompt, the right rules, the perfect context and so forth. At what point is this a roundabout way of doing it yourself anyway?
Yeah, it may feel scary but the biggest issue yet to be overcome is that to replace engineers you need reliable long horizon problem solving skills. And crucially, you need to not be easily fooled by the progress or setbacks of a project.
These benchmark accomplishments are awesome and impressive, but you shouldn't operate on the assumption that this will emerge as an engineer because it performs well on benchmarks.
Engineering is a discipline that requires understanding tools, solutions and every project requires tiny innovations. This will make you more valuable, rather than less. Especially if you develop a deep understanding of the discipline and don't overly rely on LLMs to answer your own benchmark questions from your degree.
I think OpenAi will drop their ambition for AGI and focus on product, they'll never state this of course, but it's clearly telegraphed in this for profit move.
Research and safety have to take a backseat for cost reduction. I mean there are many avenues to profitability for them, one I can think of is they could cut their cost significantly by creating smaller and smaller models that match or nearly match Gpt4, while paid subscribers wouldn't be able to really tell the difference. No one is really challenging them on their benchmark claims.
I think their main challenge is that in 5-10 years from now, if their current definition of AGI is still elusive, models of Gpt4 capabilities or similar (Llama 3 can fool most people I think) will be running locally and freely on pretty much any OS of choice without having to make a single outside API call. Every app will have access to local inference that neither costs developers nor the users anything to use. Especially after the novelty has worn off a bit, it's hard to see consumers or developers paying up to use something that's technically better but not significantly enough to justify a $20+/mo subscription or per token cost. Right now though, local inference has a huge barrier of entry, especially when you think across platforms.
Honestly, I think Google and Apple can afford to spend the cash to develop these models in perpetuity, while OpenAi needs to worry about massive revenue growth for the next few years, and they probably don't really have the personnel to grow revenue aggressively either. It's a research lab. The downside of revenue seeking too, is that sometimes the pursuit kills the product.
I took Apple how many years to change their bass config memory from 8 GB to 16? Somewhere between 8 and 10..
Regardless, I’m not sure running reasonably advanced models locally will necessarily become that common anytime soon on mainstream devices. $20 per month isn’t that much compared to the much higher HW costs, of course it’s not obvious that OpenAI/etc. can make any money longterm by charging that.
I think at its core it's not that there isn't value or future value, but currently there is an assertion, maybe some blind faith, that it's inevitable that a future version will deliver a free lunch for society.
I think the testimonies often repeated by coders that use these code completion tools is that "it saved me X amount of time on this one problem I had, therefore it's great value". The issue is that these all fall into a research of n=1 test subjects. It's only useful information for the subject. It appears we don't realize in these moments that when we use those examples, even to ourselves, we are users reviewing a product, as opposed to validating if our workflow is not just different but objectively better.
The truth lies in the aggregate data of the quality and crucially the speed by which fixes and requirements are being implemented at scale across code bases.
Admittedly, a lot of code is being generated, so I don't think I can say everyone hates it, but until someone can do some real research on this, all we have are product reviews.
> I think at its core it's not that there isn't value or future value, but currently there is an assertion, maybe some blind faith, that it's inevitable that a future version will deliver a free lunch for society.
Except in the case of "AI" we get new releases that seem somewhat impressive and therefore extend the duration for which the inflated expectations can survive. For what it's worth, stuff like this is impressive https://news.ycombinator.com/item?id=41693087 (I fed my homepage/blog into it and the results were good, both when it came to the generated content and the quality of speech)
> The truth lies in the aggregate data of the quality and crucially the speed by which fixes and requirements are being implemented at scale across code bases.
Honestly? I think we'll never get that, the same way I cannot convincingly answer "How long will implementing functionality X in application Y with the tech stack Z for developer W take?"
We can't even estimate tasks properly and don't have metrics for specific parts of the work (how much creating a front end takes, how much for a back end API, how much for the schema and DB migrations, how much for connecting everything, adding validations, adding audit, fixing bugs etc.) because in practice nobody splits them up in change management systems like Jira so far, nor are any time tracking solutions sophisticated enough to figure those out and also track how much of the total time is just procrastination or attending to other matters (uncomfortable questions would get asked them, way too metrics would be optimized for).
So the best we can hope for is some vague "It helps me with boilerplate and repeatable code which is most of my enterprise CRUD system by X% and as a result something that would take me Y weeks now takes me Z weeks, based on these specific cases." Get enough of those empirical data points and it starts to look like something useful.
I think lots of borderline scams and/or bad products based on overblown products will get funded but in a decade we'll probably have mostly those sticking around that have actual utility.
The top comment of your HN link is exactly the issue at hand
> don't know what I would use a podcast like this for, but the fact that something like this can be created without human intervention in just a few minutes is jaw dropping
AI has recently gotten good at doing stuff that seems like it should be useful, but the limitations aren’t obvious. Self driving cars, LLM’s, Stable Diffusion etc are awesome tech demos as long as you pick the best output.
The issue is the real world cares a lot more about the worst outcomes. Driving better than 99% of people 24/7 for 6 months and then really fucking up is indistinguishable from being a bad driver. Code generation happens to fit really well because of how people test and debug code not because it’s useful unsupervised.
Currently balancing supervision effort vs time saved depends a great deal on the specific domain and very little about how well the AI has been trained, that’s what is going to kill this hype cycle. Investing an extra 100 Billion training the next generation of LLM isn’t going to move the needles that matter.
Especially when you separate the ethereal "hard problems" from every day queries local LLMs can answer equally as well as SOTA models, the value proposition for these expensive models plummets. If it can't solve real hard, long horizon problems the 10% lift on a given benchmark is not a material value prop to the end user to choose a local free version over the API costs or the monthly subscription.
I think the primary reason is that datasets contain a lot more average/bad code than exceptional, and to add to that problem judging between those is possibly a subjective issue.
Developers using AI will get mostly average solutions faster but exceptional ones will be obviously rare. And, crucially if the idea itself is average or bad there isn't much an elegant coding solution will do for the idea.
I think this ultimately is the divide between the hype and reality of how AI will impact products. If you just give a product manager the keys to do all the coding as no code "prompt engineer", more than likely will lead to further enshitification of features in products with unmaintainable code bases. At the current state, understanding algorithms and thinking computationally is a requirement to improve a code base.
The hopes of having a "build me a $1 billion app" prompt capability, or "improve my shitty app" are too long horizon and subjective requests to bypass the hardships of product ideation and iteration to have the LLM deliver on the requests. It's not magic, it's probability. Averages are the end goal here, not excellence.
If we arrive at a point where LLMs translate general prompts into idealistic versions that are more like version 100 of the idea while still capturing the user's intent, then we will see these improvements. Otherwise it's copy pasta on steroids, and done mindlessly, will mostly lead to enshitification rather than improvements.
Monopolies form easily? That's funny, you should try and start one, seems quite profitable.
Seriously though, this is an oft repeated fallacy, and frankly irrelevant to the discussion.
IP laws are the actual culprit in facilitating the apparatus of the state for the creation of monopolies. Most people seem to embrace this double-think that IP laws are good while monopolies are bad. You simply don't get monopolies without IP laws. IP laws are the ultimate king maker and exclusively exist to perpetuate profits of the IP owner.
If your proposition of regulation is to disband the patent offices and repeal the copyright act, my sincere apologies.
The truth is, if you are in the position to make the step towards becoming a monopolist especially in a new market it is not impossible to do so (and by the rules it should be).
Getting to that position isn't easy tho.
But from a consumer standpoint the only thing that matters is if you have monopolists or not — we don't care how hard it was for them to become one other than it might change the number of monopolists that force their crop down our throats.
Without imaginary property, AMD would have signed a similar contract - they would rather focus on their own products rather than reverse engineering the HDMI standards to create their own implementation. At which point AMD would be in the same position, unable to reverse engineer HDMI or adopt solutions from other companies who did.
Imaginary property laws most certainly encourage and facilitate monopolies and collusion, but they are not necessary to the dynamic. Such laws are essentially just the norms of business that companies would be insisting on from other businesses anyway, for which it's much more lucrative to assent and go along with rather than attempt to defect and go against them.
Another example of this effect is the DMCA - the tech giants aren't merely following its process verbatim, but rather have used it as basis for their own takedown processes with electively expanded scope - eg why we see takedown notices pertaining to "circumvention" code, or the complete unaccountability of Content ID. Google and Microsoft aren't significantly hurting themselves by extralegally shutting down a tiny contingent of their customers, meanwhile the goodwill they garner from other corporations (and possible legal expenses they save) is immense. The loser is of course individual freedom.
The invisible hand of the free market will come and fix all the things! /s
If you talk to people who still subscribe to that notion, it quickly becomes clear that they value their miniscule chance to win the capitalist lottery more than the wellbeing of the many — the idea that markets balance everything to the advantage of everybody then seems to be just an excuse to be egoistic and without any care for others.
Don't get me wrong, nobody has to care for others and I am not going to be the person to force you, but if you don't care about others please stop pretending you are doing it for the greater good.
You're conflating several schools of thought. Utilitarianism, which appears to be your basis for defining ethical behavior, underlies this reasoning behind compulsory government action.
This line of thinking is often repeated in election cycles and mindless online discussions, with mantras like "We justify doing something heinous because it serves 'American Interests'" or "We'll coercively tax one group and redistribute funds to another because they'll do something dubiously for the 'greater good'".
However, Utilitarianism is not a foundational principle of libertarian ideology. In fact, libertarianism often refutes and rejects it as applied to governments. It doesn't prioritize egalitarianism or rely on public opinion when defining citizens' rights.
The argument for a free market unencumbered by protectionist policies isn't about the greater good; rather, it's an argument for an ethical government grounded in first principles.
The "greater good" argument tends to crumble under close examination and logical scrutiny. Its claims on reason collapse as soon as you scrutinize them more deeply.
Notably, Utilitarianism has been the basis for nearly all modern-day dictatorships, which rely on a monopoly of violence to enforce the "greater good".
It's possible to support free markets while still caring for others – this is called altruism. It's similar to utilitarianism but without coercion and fallacies.
I studied philosophy and ethics so you can safely assume I know my definitions. But that does not matter, as you apparently failed to read what I wrote.
Could you please paraphrase my "greater good argument" that crumbles under close examination? A examination you somehow failed to provide? Maybe you hoped people are too impressed by you use of words to recognize that you even failed to provide an argument against an strawman you created?
No offense, but the way you write makes you sound like a 15 year old teenager that figured out using smart words makes you sound smart, without any deeper understanding of or regard for the concepts at hand or the arguments made. If you want to show some argument is wrong you can't just simply claim it is, you need to demonstrate it - ideally using the very logic and examination, you seem to so highly value.
My original post was intended to clarify why I believe Libertarian ideology is distinct from and incompatible with Utilitarianism, particularly since in your response, you conflated the concept of the greater good as a core principle of Libertarian ideology. This is quite surprising given your claim to have "studied philosophy and ethics".
To address this misunderstanding, let me break down the logical fallacies I alluded to earlier:
- The "tyranny of the majority" problem: Since happiness is determined by the number of individuals, a simple majority can impose its will on the minority, potentially denying them their rights or freedoms.
- The "moral arithmetic" fallacy: This assumes that individual well-being can be measured and added up like numbers in an equation, ignoring the complexities of human experience and the difficulties of making such calculations.
- The "majority rules" fallacy: This implies that whatever the majority wants is automatically just or right, without considering the potential for mob rule, manipulation, or coercion.
- The "ignore individual rights" fallacy: By prioritizing the greater good over individual interests, Utilitarianism may lead to the trampling of human rights and dignity.
No offense, but it's worth noting that a more nuanced understanding of philosophy and ethics might be beneficial for more accurate representations of complex concepts.
I will defend utilitarianism, since I like it a lot and all your arguments against it are bad.
- The "tyranny of the majority" problem is a problem of direct democracy, not utilitarianism. Happiness in utilitarianism is determined not by a number of individuals, but by all individuals and perfect utility function must take into account both majorities and minorities and create consensus. This will only fail if majority and minority have directly opposed interests, but in this case overall good is still better this way (you don't want to deny majority people their rights too in favor for minorities).
- The "majority rules" fallacy is a problem of democracy overall. Every democracy system is vulnerable to this, not only utilitarianism. But then again, perfect utility function should take into account people's desire to not be fooled, so there's that.
- The "ignore individual rights" fallacy is the same as "tyranny of the majority". Utility function takes into account interests of all individuals and tries to create the best possible consensus.
- The "moral arithmetic" fallacy is the best one here, since it's actually close to the truth. You can't really create a perfect utility function, but you don't need to. You can create imperfect one and improve it later with feedback and democracy mechanisms. With time imperfect utility function will get closer and closer to perfect one. Profit maximizing utility function can't be calculated too, but corporations handle it just fine. But if you're not blind, you can see that profit maximizing utility function leads to a lot of real people suffering (climate change, wars, hunger, poverty and many many more) while leading to profit maximization (alignment problem).
Again: explain which argument about the greater good I supposedly made.
Ideally before you go off on a totally unrelated tangent again. Not trying to be mean here, but if you want others to understand why I am wrong a good start is to explain what my argument was.
Because it certainly wasn't: "conflating the concept of the greater good as a core principle of Libertarian ideology". But maybe to the reader your amount of projection onto my very simple statement is in itself telling.
"the idea that markets balance everything to the advantage of everybody then seems to be just an excuse to be egoistic and without any care for others."
There are two problems here:
1. You misstate and mischaracterize free-market ideology as having the pretense of being to the "advantage of everybody". It's potentially a byproduct but definitely not a first principle.
2. You cast a judgment of value on egotism and selfishness as being the true motivators behind free market proponents. Selfishness and egotism are human characteristics expressed across all ideological spectrums.
"Don't get me wrong, nobody has to care for others and I am not going to be the person to force you, but if you don't care about others please stop pretending you are doing it for the greater good." - Here is where you conflate utilitarian with libertarian ideology, especially as you label those who disagree with your view as pretenders and posers for the greater good, again misstating the position of your ideological opponent and then proceeding to cast a judgment of value on the positions they don't actually hold.
Not trying to be mean here, but have you thought about getting some reading comprehension lessons? It could really help you understand the things that you read as well as give you a more well rounded view things.
Have you ever considered I was talking about specific individuals that muttered those things towards me instead of reading everything I did as a paragraph from a political reader? I have no close relationship with Libertarianslism, as where I come from it is not very wide spread as a political ideology and more of a curiosity that gets mentioned at the fringes.
So what I criticized here are the things people told me in online discussions as a defense for why the system we have is okay. I did not ask them which ideology they subscribe to, but I am pretty sure that was not some pure text book form of Libertarian ideology. So I am still curious how my criticism of an observed phenomenon made you jump directly in defense of Libertarian ideology, that I neither thought about nor mentioned.
Additionally: I can start to understand what you're talking about once you start at the beginning instead of diving straight into some sort of convoluted US-internal political debate. Rephrasing what you thought the other person said and why precisely it is wrong is a good habit to keep before writing hundreds of lines attacking them on what you think they said.
Ok. This is even worse. You shouldn't use your misunderstandings from previous discussions with other people and make generalizations with everybody else you meet on new discussions, especially if you are using an incendiary tone.
I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, even while they wax poetic about scaling laws, they know there is a bottleneck ahead.
Notice how LeCun is the only researcher being honest about this in a public fashion. Meta is committed to AI already and will at least match the spend of competitors anyway, so he doesn't have as much pressure to try and convince investors that this rabbit whole is deeper.
Don't get me wrong, LLMs are a tremendous improvement on knowledge compression and distillation, but it's still unreliable enough that old school search is likely a superior method nonetheless.
Put aside consciousness or hype or investment. Look at the results; LLMs are well beyond old-school search in many ways. Sure, they are flawed in someways. Previous paradigms for search, were also flawed in their own ways.
Look at the arc of NLP. Large language models fit the pattern. One could even say that their development (next token prediction with a powerful function approximator) is obvious in hindsight.
Honestly I don't disagree, I just think that humans tend to anthropomorphize to such a high extent that there is a fair bit of hyperbole promoting LLMs as more than they are. It's my opinion that the big flaws LLMs currently present aren't going to be overcome by scaling alone.
Scaling existing architectures (inference I mean) will probably help a lot. Combine that with better training and hybrid architectures, and I personally expect to see continued improvement.
However, given the hype cycle,
combined with broad levels of ignorance of how LLMs work, it is an open question if even amazing progress will impress people anymore.
I'm less concerned with people's perception and strictly concerned with value. If we were to define value as the number of things that can be automated or severely improved by the technology or its future versions, there is a misalignment between value and perceived value.
The value is lower than perceived because there is an assumption that what's preventing higher value delivery from the investment is for the models to get better at generating responses from prompts. But there are two issues with this position.
1. LLMs still require plenty of assistance where they are writing production ready code and making function-calls, especially if the original API wasn't designed with LLMs in mind. Unless there is a leap in architecture that's going to make all the tools we have, including non-API ones easily accessible by the models to interact with, the amount of glue code required to make it all work increases the potential for features that users can use, but not necessarily deliver higher value to developers in that a lay person still probably can't develop software with an LLM copilot in toe. So yes, no one is going to be impressed even if we see models improve further on benchmarking.
2. Long-Horizon goals. Long-term research or even project management requires interdisciplinary understanding of how all the goals around success relate to each other and most importantly how to assess if an outcome is leading to a goal accomplishment or not. There isn't an architectural foundation for the models to be grounded in a reality that presupposes these abilities.
What I fail to see, is how improving next token prediction will materially move the needle on these other aspects of intelligence that aren't necessarily related to generating an output or a series of outputs orchestrated over an evolving set of requirements.
Honestly lI think that the LLM portion of the human brain has likely be surpassed by existing models
I don't think LLM for coding productivity is all hype but I think for the people who "see the magic" there are many illusions here similar to those who fall prey to an MLM pitch.
You can see all the claims aren't necessarily unfounded, but the lack of guaranteed reproducibility leaves the door open for many caveats in favor of belief for the believer and cynicism for everybody else.
For the believers if it's not working for one person, it's a skill issue related to providing the best prompt, the right rules, the perfect context and so forth. At what point is this a roundabout way of doing it yourself anyway?