Lol, you're probably not wrong. But have you ever noticed that the most important papers tend to be on the clear and readable side of things? It's as if researchers understand that being understood is important, but deemphasize that when the paper itself isn't important in the first place. (Maybe if they're only publishing to not perish, not being understood is actually a goof thing from their perspective?)
you're probably overcomplicating it; as the paper says, it's embarrassingly simple: given a problem set, generate a response for each problem with a fixed temperature and truncation - then fine tune the model on the generations.
Their hypothesis as to why this works requires a bit more knowledge about model architecture, but basically when a model generates code some positions have only one right answer and some have many valid options - but the model has to use one global confidence setting for both. Sampling with a specific temperature + a garbage-token filter, then training on those outputs, teaches the model to internalize 'be precise where there's one answer, stay open-minded where there are several' — without anyone labeling which is which.
Note that there's a lot more nuance to this and I simplified a lot.
Too true. I won't say who it was but a prominent partner in my batch referred to, essentially, a lack of morality as a "competitive advantage". I went back to the east coast after lol
this is really mind-boggling to me as someone who grew up on the (old) internet.
I think the reward factor is also a large part of it, for most of the last 10 years young people have seen that unethical behaviour results in success. For a developing brain, it's easy to see how that resulted in the current state of SV.
it's definitely a little of both. Founders my age (18-25 range) have spent the last 10 years of their life seeing that morally reprehensible behaviour is rewarded. Whether it be Trump, Musk, whoever - the reward circuit in their brain sees that being a scumbag results in success. The people who don't act that way keep their mouth shut or get publicly executed (metaphorically). It's funny that people still criticize Jobs for being hard when he was 10x a better person than 99% of AI founders.
the behaviour that was one socially unacceptable in Silicon Valley has become mainstream. The woke stuff went way too far and, like everything, the pendulum swung back the other direction with equal force. VCs are largely the problem as they set the tone. I have a lot of personal acquaintances that work in VC, and legitimately all of them fall on the spectrum of morally dubious to outright reprehensible.
reply