It was a city half in / half out of agriculture though.
No evidence of cultivation, but extensive evidence of cereal / grain processing - surrounded as it was by abundant wild grasses and steppes.
The argument made by some is that processing grain (winnowing, grinding with stone, ovens, etc) induces a fixed "city" life via the not especially portable capital investment.
Certainly an avenue of thought worth investing time in.
This comment is rather uninformed. Greco-Buddhist art was real, yes, but emerged far away from the geologic 'West' -- in Bactria, at the edge of India, where Hellenic garrisons had established a kingdom after the collapse of Alexander the Great's empire. We have very little evidence of feedback from there to the west, and so little potential for Buddhism to influence the development of later Greek society or e.g. Rome.
The most we have are faint gestures at how some Buddhists could indeed have been alive in the same place, at the same time, as early Christians. But Christ himself, as far as we know, never referenced Buddha or any Buddhist works, never interacted with a practitioner of the faith (or even referred to one), and all the same applies to the Apostles.
E.g. nobody would suggest that Buddhism was particularly influenced by Greek religious thinking, but we know with certainty that Greeks were present in the region due to their service in the Persian empire of the time.
The last line of GP's comment is key here: "Who do I sue if Palantir decides I am an illegal?"
This shouldn't make as much of a difference as it does, but due to how our legal system works, it's much harder to get meaningful legal satisfaction when an algorithm (or other inhuman distributed system) commits a crime against a person than when a person does so.
I think you're confused about the mechanism involved. It's hard to get satisfaction due to e.g. qualified immunity. The fact they use technology is largely irrelevant. You couldn't sue the NSA for spying on you before AI either.
tokens will stop being given away for free at some point, writing software was always a pretty simple white collar job, so it makes sense it's one of the earlier ones to be automated, but really the axis of evil has it's shot now at ruling the world or whatever now, but if they miss it they will eventually be subject to the market and you really will need to automate a lot more than just software developers for models this large to be worth the cost.
Of course we should really be talking about using the state or otherwise to make training larger and larger models impossible. It's not in the public good if LLMs actually get good enough to replace a lot of human labor, only a small handful of billionaires and their cronies will ever benefit from that. The Luddites were not wrong after all.
No, they didn't. It was straightforwardly unsafe and broken, the heaps of effort that went into supporting it were largely just to paper over that fact. It's no accident that the other browser vendors went along with dropping support so quickly after Apple did.
The reason given for blocking Flash on iOS at the time was it's too cpu intensive on mobile, which impacts battery life. Not that it was "unsafe and broken".
The main reason other browsers stopped supporting Flash was websites stopped being built with Flash because iOS didn't support it, and a lot of people thought that mattered even though iOS had (and still has) a small market share world-wide.
> He cited the rapid energy consumption, computer crashes, poor performance on mobile devices, abysmal security, lack of touch support, and desire to avoid "a third party layer of software coming between the platform and the developer".
Sure, he's laying out a case for the app store they'd later introduce, but it wasn't simply CPU and battery. There's a reason I cited crash logs as the primary thing I remembered about how it affected me. It gave me an immediate reason to share with people about why I couldn't fix Safari crashes when Flash was involved, which made that aspect of my job easier to explain.
If it was a traveler's union, maybe. Cop unions don't result in better outcomes for the general public, and there's no reason a controller's union won't end up just boosting pay and having a rubber room for hacks (referencing NYC schools paying teachers to not work because they're either predators or terrible at teaching, but being unable to fire them).
I wouldn't rely on a single human being to fix this kind of issue. It's only solvable through massive collaboration and communication among those who want to fix it.
Movements that ignore the need for a charismatic leader fail, often spectacularly. It's why for example occupy wallstreet was such a laughable failure. Who was its leader? Is the human megaphone a species of "massive collaboration and communication"? Can you name me one leader from that movement who was nationally recognized as such?
Strong leaders are always required. Such people reduce the cost of messaging and communication which would otherwise be insurmountable to cohere a movement and actually make change. You don't elect a mob. Find leaders you trust and spread your conviction without apology. Roosevelt was not Roosevelt until after his works were done. We don't need some amorphous "massive collaboration and communication" we need to elect leaders who will fight for what we believe. So many of your friends, family and neighbors are willing to elect sell-out leaders. You could start there, that is if you actually want to fix the problem rather than invent new ones.
> It's why for example occupy wallstreet was such a laughable failure.
This claim is enormous. I would instead argue that the movement lacked cohesiveness because it basically complained about too large a set of (correctly identified as interconnected) issues and lost momentum because the surface was too large.
That said, I agree w your point about a face being important. Even in software, where tech can speak for itself, we see this heavily: Torvalds, Matsumoto, van Rossum, Jobs,
...which is typically done by building a movement around a leader who represents the values a movement wants to achieve.
FDR is a good example of an American leader who made substantive, wildly successful, left-leaning policy changes that ushered in decades of prosperity and (in part) last to this very day despite facing heavy opposition from the business elite of the time. They even tried to coup him!
At the time, the long term trends were dire for the American left. Double insulation was strong and getting stronger. Then the Great Depression hit. Around the world, populists and radicals were elected to office, and one way or another they changed things. In America, we managed our reform process without trying to conquer the world and without starving millions. Not Hitler, not Stalin. Roosevelt. I think that's a worthy goal to aim for again this time around.
Perhaps I mean to ask a question then, how did FDR manage to become such a widely heard leader back then with so many less ways for people to talk together? Did it make a bigger difference that he had to exist as someone people spoke to other people about? Shouldn't it be easier to find these leaders with so much more access to everyone nowadays?
Communication friction is only one cost of running a campaign among many, so the structure of parties and campaigns and primary / general elections has largely remained the same. Even if the technological barriers went away, I suspect the human factors would still hold up the structure because only so many people are willing to spend years of their life building legitimacy and promoting a political platform and each voter is only willing to spend a certain amount of time participating and choosing.
Exactly how that may have played out in the last century could be explained by many, many chains of causes and effects. But it wasn't a great leader that made it happen. At the bottom of everything, I believe it was this:
Decades of Famine, Pestilence, War, and Death destroyed not only capital but huge swaths of the labor pool. With labor at a premium, it became more valuable and power shifted.
I think that without a similar apocalypse, it will not happen again.
Yes, economic disaster is the driver (tangential: a lump-of-labor supply shock was not the transmission mechanism), but big political movements always happen from the pieces lying around. Everyone can feel that a disaster of one form or another is coming. We need to make sure the right pieces are lying around.
Out of curiosity, what sort of things have you seen it do that better fit 'autoresearch' than 'autotune' thus far? Optimizations it made that wouldn't be been surfaced by an autotune system, I suppose.
The most recent round of autoresearch (round 2) which decreased "time to GPT-2" from 1.8 hours to 1.65 hours had some examples. I adjusted the program.md to "look at modded nanogpt project and draw inspirations from there for things to try" and it came back with a bunch of tuning, but also tried and implemented new architecture changes, some of which actually helped including the smear gate and the backout skip connection. These are not just hyperparameters, they are new PyTorch code. I'm now working on a more general system that can have a queue of ideas that could be sourced from archive papers, github repos, etc.
Do you have a sense of whether these validation loss improvements are leading to generalized performance uplifts? From afar I can't tell whether these are broadly useful new ideas or just industrialized overfitting on a particular (model, dataset, hardware) tuple.
Did you consider providing the LLM with a framework for automatic hyperparamter tuning? This would free up its capacity to focus on the more important architectural decisions.
I see this critique about autoresearch online often, but I think it’s misplaced.
Here’s a use case that may illuminate the difference, from my own work at Nvidia. Im currently training some large sparse autoencoders, and there are issues with dead latents. Several solutions exit to help here, such as auxk, which I can certainly include and tune the relevant params as you describe. However, I have several other ideas that are much different, each of which requires editing core code (full evaluation changes, initialization strategies, architecture changes, etc.), including changes to parallelism strategies in the multi-rank environment I’m using. Moreover, based on my ideas and other existing literature, Claude can try a number of new ideas, each potentially involving more code changes.
This automated run-and-discover process is far beyond what’s possible with hyperparam search.
It wasn't meant as a critique, I'm legitimately interested in knowing more about where it can push boundaries and where it struggles. I agree that in general it's a truism that "Claude can try a number of new ideas" etc., but the question remains as to where in particular it actually takes advantage of this to push the envelope in a way other tools don't -- since that informs when it makes sense to use something like this.
reply