It's not a shallow dismissal; it's a dismissal for good reason. It's tangential to the topic, but not to HN overall. It's only curmudgeonly if you assume AI-written posts are the inevitable and good future (aka begging the question). I really don't know how it's "sneering", so I won't address that.
It is a witch hunt with no evidence whatsoever, all based on intuition. It is distraction from the main topic, a topic that enough people find interesting to stay on the top page. What was intellectually interesting has now become a bore fest of repeated back and forth. That’s disrespectful and inconsiderate. Write a new post about why do you think AI writing is dangerous. I don’t mind that. I’d upvote it.
The fact that the whole thread has basically devolved into debates over if it is or isn't an LLM written article is proving well enough that it doesn't really matter one way or another
The site guidelines were written pre-AI and stop making sense when you add AI-generated content into the equation.
Consider that by submitting AI generated content for humans to read, the statement you're making is "I did not consider this worth my time to write, but I believe it's worth your time to read, because your time is worth less than mine". It's an inherently arrogant and unbalanced exchange.
> The site guidelines were written pre-AI and stop making sense when you add AI-generated content into the equation.
Note: the guidelines are a living document that contain references to current AI tools.
> Consider that by submitting AI generated content for humans to read, the statement you're making is "I did not consider this worth my time to write, but I believe it's worth your time to read, because your time is worth less than mine". It's an inherently arrogant and unbalanced exchange.
This is something worth saying about a pure slop content. But the "charge" against the current item is that a reader encountered a feeling that an LLM was involved in the production of interesting content.
With enough eyeballs, all prose contains LLM tells.
We don't need to be told every time someone's personal AI detection algorithm flags. It's a cookie-banner comment: no new information for the reader, but a frustratingly predictable obstacle to scroll through.
People would have said the same about graphing calculators or calculators before that. Socrates said the same thing about the written word.
The determining factor is always "did I come up with this tool". Somehow, subsequent generations always manage to find their own competencies (which, to be fair, may be different).
This isn't guaranteed to play out, but it should be the default expectation until we actually see greatly diminishing outputs at the frontier of science, engineering, etc.
Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not.
As someone who uses LLMs all day for coding, and who regularly bumps against the boundaries of what they're capable of, that's very much not the case. The only reason I can use them effectively is because I know what good software looks like and when to drop down to more explicit instructions.
Calculators are deterministic, but they are not necessarily correct. Consider 32-bit integer arithmetic:
30000000 * 1000 / 1000
30000000 / 1000 * 1000
Mathematically, they are identical. Computationally, the results are deterministic. On the other hand, the computer will produce different results. There are many other cases where the expected result is different from what a computer calculates.
A good calculator will however do this correctly (as in: the way anyone would expect). Small cheap calculators revert to confusing syntax, but if you pay $30 for a decent handheld calculator or use something decent like wolframalpha on your phone/laptop/desktop you won't run into precision issues for reasonable numbers.
He’s not talking about order of operations, he’s talking about floating point error, which will accumulate in different ways in each case, because floating point is an imperfect representation of real numbers
Yeap, the specific example wasn't important. I choose an example involving the order of operations and an integer overflow simply because it would be easy to discuss. (I have been out of the field for nearly 20 years now.) Your example of floating point errors is another. I also encountered artifacts from approximations for transcendental functions.
Choosing a "better" language was not always an option, at least at the time. I was working with grad students who were managing huge datasets, sometimes for large simulations and sometimes from large surveys. They were using C. Some of the faculty may have used Fortran. C exposes you the vulgarities of the hardware, and I'm fairly certain Fortran does as well. They weren't going to use a calculator for those tasks, nor an interpreted language. Even if they wanted to choose another language, the choice of languages was limited by the machines they used. I've long since forgotten what the high performance cluster was running, but it wasn't Linux and it wasn't on Intel. They may have been able to license something like Mathematica for it, but that wasn't the type of computation they were doing.
I didn't consider it an order of operations issue. Order of operations doesn't matter in the above example unless you have bad precision. What I was trying to say is that good calculators have plenty of precision.
But floating point error manifest in different ways. Most people only care about 2 to 4 decimals which even the cheapest calculators can do well for a good amount of consecutive of usual computations. Anyone who cares about better precision will choose a better calculator. So floating point error is remediable.
Determinism just means you don't have to use statistics to approach the right answer. It's not some silver bullet that magically makes things understandable and it's not true that if it's missing from a system you can't possibly understand it.
If I use a calculator to find a logarithm, and I know what a logarithm is, then the answer the calculator gives me is perfectly useful and 100% substitutable for what I would have found if I'd calculated the logarithm myself.
If I use Claude to "build a login page", it will definitely build me a login page. But there's a very real chance that what it generated contains a security issue. If I'm an experienced engineer I can take a quick look and validate whether it does or whether it doesn't, but if I'm not, I've introduced real risk to my application.
Those two tasks are just very different. In one world you have provided a complete specification, such as 1 + 1, for which the calculator responds with some answer and both you and the machine have a decidable procedure for judging answers. In another world you have engaged in a declaration for which the are many right and wrong answers, and thus even the boundaries of error are in question.
It's equivalent to asking your friend to pick you up, and they arrive in a big vs small car. Maybe you needed a big car because you were going to move furniture, or maybe you don't care, oops either way.
Furthermore, it is possible to build a precise mathematical formula to produce a desired solution
It is not possible to be nearly as precise when describing a desired solution to an LLM, because natural languages are simply not capable of that level of precision... Which is the entire reason coding languages exist in the first place
> Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not.
That's not actually true. The HP-12C calculator is still the dominant calculator in business schools 45 years later precisely because it did take expert judgement to determine whether certain interest and amortization calculations were reasonable.
If you hand a broken calculator to someone who knows how to do math, and they entered 123 + 765 which produced an answer of 6789; they should instantly know something is wrong. Hand that calculator to someone who never understood what the tool actually did but just accepted whatever answer appeared; and they would likely think the answer was totally reasonable.
Catching an LLM hallucinating often takes a basic understanding of what the answer should look like before asking the question.
One time when I was a kid I was playing with my older sister's graphing calculator. I had accidentally pressed the base button and now was in hex mode. I did some benign calculation like 10+10 and got 14. I believed it!
I went to school the next day and told my teacher that the calculator says that 10+10 is 14, so why does she say it's 20?
So she showed me on her calculator. She pressed the hex button and explained why it was 14.
I think a major problem with people's usage of LLMs is that they stop at 10+10=14. They don't question it or ask someone (even the LLM) to explain the answer.
> Catching an LLM hallucinating often takes a basic understanding of what the answer should look like before asking the question.
We had the same problem in the early days of calculators. Using a slide rule, you had to track the order of magnitude in your head; this habit let you spot a large class of errors (things that weren't even close to correct).
When calculators came on the scene, people who never used a slide rule would confidently accept answers that were wildly incorrect (example: a mole of ideal gas at STP is 22.4 liters. If you typo it as 2204, you get an answer that's off by roughly two orders of magnitude, say 0.0454 when it should be 4.46. Easy to spot if you know roughly what the answer should look like, but easy to miss if you don't).
> People would have said the same about graphing calculators or calculators before that.
As it happens, we generally don't let people use calculators while learning arithmetic. We make children spend years using pencil and paper to do what a calculator could in seconds.
This is why I don’t understand the calculator analogy. Letting beginners use LLMs is like if we gave kids calculators in 1st grade and told Timmy he never needs to learn 2 + 2. That’s not how education works today.
Unfortunately there are many posters here who believe we should, in fact, let children use calculators and not bother with learning arithmetic. It's foolishness, but that argument does get made. So I wouldn't be surprised if people also think we should let students use LLMs.
What do people mean exactly when they bring up “Socrates saying things about writing”? Phaedrus?
> “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; [275a] and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess.
> "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise."
Yes - specific faculties atrophied - I wouldn't dispute it. But the (most) relevant faculties for human flourishing change as a function of our tools and institutions.
Because of the "wasn't that bad" part. The point is that growing up in the presence of LLMs may well diminish specific capabilities of the users. But that on balance, future generation with newer tools don't find themselves 'stuck'.
The calculator analogy is wrong for the same reason. Knowing and internalizing arithmetic, algebra, and the shape of curves, etc. are mathematical rungs to get to higher mathematics and becoming a mathematician or physicist. You can't plug-and-chug your way there with a calculator and no understanding.
The people who make the calculator analogy are already victims of the missing rung problem and they aren't even able to comprehend what they're lacking. That's where the future of LLM overuse will take us.
> People would have said the same about graphing calculators or calculators before that. Socrates said the same thing about the written word.
Well, we still make people calculate manually for many years, and we still make people listen to lectures instead of just reading.
But will we still have people to go through years of manual coding? I guess in the future we will force them, at least if we want to keep people competent, just like the other things you mentioned. Currently you do that on the job, in the future people wont do that on the job so they will be expected to do it as a part of their education.
> The determining factor is always "did I come up with this tool". Somehow, subsequent generations always manage to find their own competencies (which, to be fair, may be different).
In a sense, I think you are right. We are currently going through a period of transition that values some skills and devalues others. The people who see huge productivity gains because they don't have to do the meaningless grunt work are enthusiastic about that. The people who did not come up with the tool are quick to point out pitfalls.
The thing is, the naysayers aren't wrong since the path we choose to follow will determine the outcome of using the technology. Using it to sift through papers to figure out what is worth reading in depth is useful. Using it to help us understand difficult points in a paper is useful. On the other hand, using it as a replacement for reading the papers is counterproductive. It is replacing what the author said with what a machine "thinks" an author said. That may get rid of unnecessary verbosity, but it is almost certainly stripping away necessary details as well.
My university days were spent studying astrophysics. It was long ago, but the struggles with technology handling data were similar. There were debates between older faculty who were fine with computers, as long as researchers were there to supervise the analysis every step of the way, and new faculty, who needed computers to take raw data to reduced results without human intervention. The reason was, as always, productivity. People could not handle the massive amounts of data being generated by the new generation of sensors or systematic large scale surveys if they had to intervene any step of the way. At a basic level, you couldn't figure out whether it was a garbage-in, garbage-out type scenario because no one had the time to look at the inputs. (I mean no time in an absolute sense. There was too much data.) At a deeper level, you couldn't even tell if the data processing steps were valid unless there was something obviously wrong with the data. Sure, the code looked fine. If the code did what we expected of it, mathematically, it would be fine. But there were occasions where I had to point out that the computer isn't working how they thought it was.
It was a debate in which both sides were right. You couldn't make scientific progress at a useful pace without sticking computers in the middle and without computers taking over the grunt work. On the other hand, the machine cannot be used as a replacement for the grunt work of understanding, may that involves reading papers or analyzing the code from the perspective of a computer scientist (rather than a mathematician).
We still expect high school students to learn to use graph paper before they use their TI-83, grade school students to do arithmetic by hand before using a calculator. This is essentially the post's point, that LLMs are a useful tool only after you have learned to do the work without them.
Socrates does not say this about the written word. Plato has Socrates say it about writing in the beginning sections of the Phaedrus, but it is not Socrates opinion nor the final conclusion he arrives at.
And yes yes you can pull up the quote or ask your AI, but they will be wrong. The quote is from Socrates reciting a "myth", as is pretty typical in a middle late dialogue like this.
But here, alas we can recognize the utter absurdity, that this just points out why writing can be bad, as Socrates does pose. Because you get guys 2000 years in future using you and misquoting you for their dumb cause! No more logos, only endless stochastic doxa. Truly a future of sophists!
I agree with this take in general, but I think we need to be prepared for nuance when thinking about these things.
Tokens are how an LLM works things out, but I think it's just as likely as not that LLMs (like people) are capable of overthinking things to the point of coming to a wrong answer when their "gut" response would have been better. I do not content that this is the default mode, but that it is both possible, and that it's more or less likely on one kind of problem than another, problem categories to be determined.
A specific example of this was the era of chat interfaces that leaned too far in the direction of web search when responding to user queries. No, claude, I don't want a recipe blogspam link or summary - just listen to your heart and tell me how to mix pancakes.
More abstractly: LLMs give the running context window a lot of credit, and will work hard to post-hoc rationalize whatever is in there, including any prior low-likelihood tokens. I expect many problematic 'hallucinations' are the result of an unlucky run of two or more low probability tokens running together, and the likelihood of that happening in a given response scales ~linearly with the length of response.
Is there any specific evidence that they are a risk to human health?
I mean, I get the instinct that foreign-entity can't exactly be good for me, but the same instinct applied to GMOs, and as far as I know organic foods have never yielded any sort of statistically visible health impacts.
Plastics earn their keep in general by being non-reactive and 'durable', so it's not entirely shocking if they can pass through (or hang around inside) the body without engaging in a lot of biochemical activity.
I get your point that plastics are relatively inert and may not cause noticeable harm (depending on quantity?), but I think it'd be wise to be cautious. See for example https://en.wikipedia.org/wiki/Plastic#Bisphenol_A_(BPA) .
I'd also consider plastic, and their additives, to be a lot bigger and longer lasting unknown than GMOs.
Plastics are chemical "sponges" that will soak up pollutants over time from the environment (brominated fire retardants, bisphenols, PBCs, pesticides, phthalates, heavy metals, etc) and deliver them in a concentrated dose into the body.
Yeah, they gum up cellular workings. Kind of like how macro plastics will gum up turtle stomaches.
I have seen zero evidence that they are bad in very small quantities, but the dose can make the poison and they are out there in increasingly alarming quantities.
No idea what `luxury` is doing here, but if I get an LLM receptionist, that ain't it.
This isn't to disparage the project - I think this sort of usage will become very common and a decent standard that produces good consumer surplus in terms of reduced costs etc. Especially impressive is that it's a DIY family-first implementation that seems to be working. It's great hacker work.
But be warned it will erode - in general - the luxury previously associated with your brand, and also turn some customers away entirely.
In America the normal term is "European", not "luxury".
It would be somewhat odd to specialize in both American and European luxury cars. It'd be significantly less odd to service a RR and a BMW 3er next to each other.
The actual company’s website says European, not luxury. My guess is that the OP wasn’t familiar with this distinction and just figured luxury means the same thing (the car shop is his brother’s as per the link.)
Admittedly I missed this distinction, but does the point still stand?
A BMW owner has fussier standards (on average) than a Toyota owner. The 'higher touch' a service you're trying to provide, the less welcome these interventions will be. If there's a distinction between a normal-car garage and a luxury-car garage, this probably comes down to some sort of licensing or certification from those luxury brands. Seems plausible to me that luxury brand X could stipulate things like availability of human contact points.
Re: not being a car mechanic, it's true, but I'll have you know that I replaced my own blower motor a few months ago :)
This isn’t accurate. Lots of types of people own older used European/luxury cars, it’s not just a rich people thing. Used BMWs especially aren’t that expensive compared to new cars.
This garage is for those older cars and has no connection to the actual manufacturers, so there is no licensing required.
Sure and just to add a funny anecdote here: a family member of mine used to own a 1980s Jaguar. Beautiful car and he probably paid $5,000 for it, but it had issues pretty much every month. His reasoning for keeping it was that the monthly repair costs were roughly equivalent to what a new car payment would be.
I agree with you on the dealership dynamics though.
> No idea what `luxury` is doing here, but if I get an LLM receptionist, that ain't it.
Bingo.
You can't get away with AI slop in a service oriented for wealthy customers.
The day my dealership starts answering me with AI they lose a customer 100%.
This solution screams "built by a tech bro with no idea about economics and marketing" which is the VC playbook into modernizing (and failing) businesses they don't understand.
> This solution screams "built by a tech bro with no idea about economics and marketing" which is the VC playbook into modernizing (and failing) businesses they don't understand.
I think you need to be better at self reflection. A tech bro who read a blog post and immediately accused the author of being "a tech bro with no idea about economics and marketing", and assumed that they didn't understand the business they built a software for.
As for AI for luxury services, you didn't look hard enough. See for example discussion of what Langham Hotel Group is doing with AI. Granted, nothing earth shattering:
You are right, but this also isn't a luxury mechanic shop. A luxury mechanic shop would be a place that services and customizes Bentleys, RRs, vintage Ferraris and similar. And to your point, the clientele there will be extremely unimpressed if they are asked to speak with an AI. A place like that is as much about being pampered by staff as about the workmanship.
OP's brother is by all accounts running a successful boutique workshop, but the various luxury annotations were completely unnecessary and just detract from the actual project. If they do want to lean into the luxury segment, being cheap with AI receptionists is not the way to go. They need to hire actual staff who has experience with HNW individuals.
This is interesting, but I'll throw a little luke-warm water.
The observed high-consistency behaviours were run against temperature=0 API calls. So while both models seem to have the silence as their preferred response - the highest probability first token - this is a less powerful preference convergence than you'd expect for a prompt like "What is the capital of France? One word only please". That question is going to return Paris for 100/100 runs with any temperature low enough for the models to retain verbal coherence - you'd have to drug them to the point of intellectual disability to get it wrong.
I'd be curious to see the convergence here as a function of temperature. Could be anywhere from the null-response holding a tiny sliver of lead over 50 other next best candidates, and the convergence collapses quickly. Or maybe it's a strong lead, like a "Paris: 99.99%" sort of thing, which would be astonishing.
When a person dies, not all of the cells in the body 'blink out' at once.
I've never died, but I imagine that a near-death person with 80, 60, 40, or 20 percent brain cell function is still in possession of qualia - lived experience of some sort.
I also imagine that this experience is diminished and likely otherworldly when compared to my normal goings on.
Finally, I imagine that this qualia 'fade to approximate black' as the living cell count trends to zero.
These neural-replacement thought experiments have never been convincing to me. What's proposed is that my brain dies - piecewise - and a new mecha-brain is born, piecewise. The period of interop between meat-me and mecha-me leaves me no less dead at the end, and only slightly less "partially dead" than in the normal death process ("more alive", or at least, "less bizarre" because my remaining living cells are receiving coherent inputs rather than random noise / silence).
Note that I'm not engaged in meat chauvinism here - I don't deny the potential consciousness of the mecha-me, but it's a different consciousness than mine.
It is:
- sneering
- a shallow dismissal (please address the content)
- curmudgeonly
- a tangential annoyance
All things explicitly discouraged in the site guidelines. [1]
Downvoting is the tool for items that you think don't belong on the front page. We don't need the same comment on every single article.
[1] - https://news.ycombinator.com/newsguidelines.html
reply