They are saying that comparison to other models only about the problems it was jailbroken to complete in the government's example, not all vulnerabilities it could exploit unjailbroken.
Ratified international treaties are the supreme law of the land in the US, and the whistleblower protection stuff being invoked is claimed to be from retaliation to a crime allegation.
Norbert Weiner was an atheist but he talks about three areas religion is the only thing to have really examined that relate to capable AI: omniscience, omnipotence, and worship (gadget worship). It has very prescient stuff on blackbox learning/distillation, reinforcement learning/reward hacking, alignment through human feedback.
His The Human Use of Human beings and Cybernetics are extremely good too and have more of a mash of the themes between Rerum Novarum and Magnifca Humanitas, and more near-term automation.
Yes, it starts slow with a lot of history of Cybernetics stuff through Leibniz and stuff (kind of prescient given chain rule -> back propagation, and control theory's relevance to optimizers). It is about twice as long as God & Golem and covers a lot of the same plus more examination of automation and human augmentation. I think either it or Cybernetics also goes into mass communication with some relevance to how social media played out.
God & Golem is the most succinct and up to date though, probably a 2hr read.
The book Cybernetics is a lot of math and ergodic theory stuff that went beyond me, but is the longest and you still get a lot out of it skimming over that stuff if you don't have the background for it. The last revision of it in the 50s added some of the same blackbox function copying/imitation learning/distillation stuff, reinforcement learning with reward hacking concerns, and superintelligence as genie/monkey's paw.
I would read the three in reverse order of publication.
He also foresaw another big area of potential existential danger, Wiener filter for guidance and control of missiles (later superceded with Kalman filter bringing the nuclear hard targets era with 15min retaliation windows) and refused to work on it or share prior work, and he also had bioweapons delivery concerns before the bioweapons treaties, publishing this open letter in The Atlantic in 1947:
I thought Starship has many unique tiles, they ended up needing to vary thickness to match heat patterns to save weight and have some complicated geometry near the fins.
Shuttle had more unique perimeter shapes, but starship still has a lot of variation due to tickness. I don't know if that is easier to vary in production or still needs custom molds for the variation.
Part of the counter was Microsoft was going to try and hire everyone individually, compensating for the lost stock appreciation, without the org itself, if they went through with maintaining the firing. I think that could have been much harder to pull off, but maybe they made them believe it was an inevitable outcome.
I don't know that the hypothetical I described requires that the open source part was sincere. The article focuses on Altman's belief that someone was going to develop AI and he thought it would be bad if Google did it first--meaning, by implication, that it would be good if an organization that he started did it first. But what if it turns out to be bad no matter who did it first?
To the extent that that's true, it's true because Sam Altman did it at all.
It is not that hard to construct a scenario wherein an alternative version of Sam Altman pushed really hard for a genuinely ethical LLM—one trained only on data that he had full unclouded rights to, either because it was public domain or because he had bought or otherwise explicitly received the rights, among other measures—and had then made that competitive in the market with the rest of the existing crop of LLMs. In a scenario like that, whether his LLM did well or not, one could clearly see that Alt-Altman, as it were, was genuinely making an effort.
The reality doesn't look anything like that. In this reality, it doesn't matter whether Altman thought he was trying to get there first because others would make it worse: he made it worse, and his actions match up very, very well with what one would expect the actions of a man who just wanted money, control, and power to be.
I'm not sure I agree that Altman has made the AI situation worse. I think Altman was right that Google, and Microsoft and others for that matter, were going to pursue AI no matter what. Even if OpenAI had never existed and Altman had never pursued AI at all, we'd still have much the same situation we have now.
Even if it's true that, if OpenAI had never existed and Altman had never pursued AI at all, we wouldn't have had the spectacle of a supposed nonprofit trying to do "ethical" AI and then pivoting to a for-profit company, that actually might be something that was worth learning: that it's not possible to do AI as a private nonprofit with an explicit commitment not to do certain things.
Sorry, my phrasing there was kind of poor: I didn't mean that he had made the overall AI situation worse, but rather that the AI he made was worse for the public than a hypothetical AI that was made with a deliberate eye to ethical or at least neutral behavior.
Ah, ok. I certainly agree with that. My question would be whether it's even possible for we humans as we are now to make an AI with a deliberate eye to ethical behavior. Sam Altman clearly is not someone who can do that (nor is Elon Musk, for that matter). But I'm not sure I know of anyone who is.
I mean, there are plenty of people with the ethical chops to do so, and, I daresay, plenty who combine that with the technical chops.
What's lacking is the money & power.
Having/obtaining enough of those to be able to compete with the likes of OpenAI and Google is, unfortunately, nearly guaranteed to result in a deeply ethically compromised person at best.
Earlier of their systems have solve other Erdos problems that people had worked on, this one was more monumental and had had a lot more prior effort that didn't solve, but this isn't a one-off.
This is true, but I still think the relevant question is, how many did they try before they found one that yielded to LLMs? The conclusion is very different if they tried 100 open problems and succeeded at one.
Yeah, maybe it's just the Texas Sharpshooter Fallacy basically, but with AI.
And if it isn't, we should find out very soon. If AI has got so good as OpenAI's post implies, then we should soon see a veritable blooming in the production of mathematical results, by lay people no less. No mathematicians needed! OpenAI say that their secret LLM solved the planar unit distance problem "autonomously" and the companion remarks say it one-shotted it; and while the companion remarks make it clear that there was a lot of refinement and improvement work done by humans, everyone seems to agree that the AI did the job by itself.
If that's true, if we're really at that level of autonomous mathematical reasoning ability, then we should see hundreds, even thousands, of open problems suddenly solved in a matter of years if not months. We'll just have to wait and see.
Yes, as some of these are being solved by the same person, I think my point is even more relevant: you try 1000 problems and solve a few, and only report the few, and it just seems like a matter of time until the rest are solved. But if you report that it didn’t work on the others, your conclusion is different.
I think it is important to temper expectations in light of the fact that these announcements are coming from a startup company with shady values looking to imminently IPO, and thus represent the most biased and misleading take of the situation possible.
>> If that's true, if we're really at that level of autonomous mathematical reasoning ability, then we should see hundreds, even thousands, of open problems suddenly solved in a matter of years if not months.
reply