There's definitely a sizable contingent of people who desperately want to believe consciousness is just electrical impulses in our brain. Because "what else could it be"? The fact is that we just don't know, and "abiding in the not-knowing" is for many the most uncomfortable thing ever. Especially for the curious- and rational-minded people this forum tends to attract. I'm one of them, too.
It is basically reductionism, or in the extreme atomism, and (as I understand it) it has largely fallen out of fashion both in the philosophy of science and in the philosophy of mind, since it heyday in the 1970s. And I don’t see it coming back into fashion any time soon.
I can definitely see why reductionism would appeal to an educated public. We learn about the periodic table; we are able to break sentences down to words, down to letters; to break an executable down to binary code, to machine instruction, to electrical currents flowing through semiconductors. Why shouldn’t we be able to do the same with conscious thoughts? It is certainly an appealing thought process.
As I understand it, reductionism started to fall out of favor because of the rise of quantum mechanics and chaos theory, where we have a lot of weird phenomena which cannot be explained by reducing the particles down to the sub-sub atomic (or rather they are better explained by describing the interactions directly).
Quantum Mechanics and Chaos Theory don't preclude reductionism, and I wouldn't say it has fallen out of favor as a whole. Certain types certainly have, but not the overall idea.
Also, nothing about the idea of the mind only being made up of physical processes means things have to be deterministic.
It doesn’t preclude it correct, however it provides a pretty strong examples of where reductionism is lacking, which what I believe has turned a lot of philosophers of science against pure reductionism (I am probably oversimplifying here. An expert science historian [which I am not] could probably write a whole book about why reductionism is not as popular today as it was in the 1970s).
There are a whole lot more physical processes going on in our bodies then just neural activity. And my best guess is that is exactly where reductionism fails. It is possible that neural activity is a necessary but not sufficient condition for consciousness. It is also possible that we are looking in the wrong direction, that consciousness arises via interactions with the world. In either case (of which I find the former quite convincing) we will never be able to describe the mind by just looking at neural activity.
I am actually of the opinion that cognitive scientists are doing an excellent job describing the mind with our current theoretical models which excludes the tough questions of consciousness.
I feel like these are separate things... neural activity being necessary but not sufficient for consciousness does not mean reductionism is wrong, it just means the fundamental building block is not a neuron.
It might not even be possible to fully understand the physical mechanisms that underlie consciousness, but that doesn't mean there has to be something more than physical mechanisms.
Yes. Even to this reductionist, “neural activity” is insufficient to describe to consciousness in the same way that “it’s physics” is insufficient to describe how a car works.
I could put a bunch of metal and rubber and gasoline in a pile and light it on fire — all the necessary ingredients for a car — but it wouldn’t create a working automobile. The arrangement of the objects and processes matters.
In the same way, if you put a bunch of brain cells together in a Petri dish, but their connections or firings were disordered, I wouldn’t expect consciousness. “Neural activity” is thus insufficient on its own, but this I doesn’t mean reductionism is incorrect. It just means you didn’t correctly reduce the problem to the correct constituent parts. You left some out.
Reductionism is a theoretical framework. It is neither right nor wrong, Sometimes a theory based on reductionism is wrong, but reductionism it self is never wrong.
Reductionism usually includes interactions of the lower parts (unless you are an atomist; in which case go back to ancient Greece), I never denied this. However even with the interactions, reductionism is still a lacking framework to describe consciousness. If it wasn’t so lacking it would be more popular among the people who actually study the mind.
I didn’t say it was wrong, I said it was lacking and unpopular among modern philosophers of science. If you want to explain consciousness as arising from interaction with the environment (like Ted Chiang did in yesterday’s article) holism is a much better approach, same if you want to use evolutionary explanations, like Daniel Dennett did at the turn of the century.
I think reductionism is simply to limited of a philosophical framework for modern science and philosophy.
I'm a bit late to the party, but I notice that people are missing the core argument of this article completely. That core, summarized below, seems very hard to dismiss to me. This argument has been presented before, but there seems to be something about it that makes it very hard for people to grasp.
Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of "themselves", as inferred from the training data plus the text of the conversation so far, just like they do for any other role. This is fundamentally different than we humans. When we have a (normal, non-roleplaying) conversation, we don't put aside whatever role we might have been acting out before and instead take on the role of "ourselves" and act from this new perspective. Acting as ourselves is a completely different way of interacting than any play-acting we might do for example as an actor in a play, or when trying to predict the future behaviour of a third-party. Even in terms of energy expenditure, acting as ourselves is much less strenuous than trying to simulate someone else's perspective.
IOW, we have an inherent perspective, an I like an inner eye which looks at all things with a certain slant or tendency. LLMs don't. There's no I inside them, instead they can take, and in a sense are, all Is possible.
What? Where are you seeing such claims? I.e. that only humans can be conscious, that AI in general is not conscious by definition?? Seems to me you've put up an easy-to-dismiss strawman.
Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of "themselves", as inferred from the training data plus the text of the conversation so far, just like they do for any other role. This is fundamentally different than we humans. When we have a (normal, non-roleplaying) conversation, we don't put aside whatever role we might have been acting out before and instead take on the role of "ourselves" and act from this new perspective. Acting as ourselves is a completely different way of interacting than any play-acting we might do for example as an actor in a play, or when trying to predict the future behaviour of a third-party. Even in terms of energy expenditure, acting as ourselves is much less strenuous than trying to simulate someone else's perspective.
IOW, we have an inherent perspective, an I like an inner eye which looks at all things with a certain slant or tendency. LLMs don't. There's no I inside them, instead they can take, and in a sense are, all Is possible.
>> The first requirement is that the computer program has a body (either physical or virtual) and sense organs
LLMs may be chameleons, but, to steal an excellent quote, if you can’t tell does it really matter.
My personal take on this is that consciousness is a thing induced, not a thing evoked. If it generates a response on the observer as a conscious thing would it is, by my definition, conscious.
Sure, by your definition they totally are conscious simply because at one point in the past they fooled one person that they might be so (and so is the OG chat agent, ELIZA). But no one else will accept such a definition.
And if you read my comment above carefully, I point out at least one way you could tell whether an agent has an I or just pretending to – by tracking their energy expenditure/reaction times. But they would give themselves in other ways as well. Pretending can only fool one short-term.
I'm not going to discuss the quote since @sp1nningaway already pointed out it doesn't say what you say it does.
No greed whatsoever? Unlikely. But less? Possibly, why not? There's no reason to believe greed cannot fluctuate across societies. Quite the opposite, I'd expect fluctuations along any given dimension by default.
Yes, they are exaggerations, similar things have been said about the Minoans. It's still true AFAIK that these societies were much less militaristic than the ones that followed them.
He said exactly why: because Trump's policies are unpredictable. Before that, there was no problem, really. Of course, it's a political movement and Trump is much a symptom as a disease, but you're saying we should thank him for bringing about this unpredictability — because now we can see that unpredictability is possible? There's something seriously loopy about that argument. It's like asking one to thank the burglars because they woke them from their peaceful slumber of safety...
i think you re over-indexing on Trump and not recognizing that this has been a problem for Europe for much longer than Trump has been a serious political figure. Europe has and to some extent is still very much asleep and holding on to a world that no longer exists. Hyper-focusing on Trump is a dangerous manifestation of that antiquated understanding of the world. The US is at the bottom of Europe’s list of problems.
You've just reiterated the exact same points you made in the comment I replied to. Consider that it might be you who is getting hung up* on the "Trump" name, which, in common discourse, can, and usually does function as a signifier not just for the man himself but for the Trumpist political attitude and movement, and even more generally, the quasi-fascist/quasi-monarchist regressive politically infantile post-truth nationalist/authoritarian-revival movements which we are witnessing world-wide. Declaring that this phenomenon is "at the bottom of Europe's list of problems" might sound like surprising and important, but it's just plain false.
As for Europe being "asleep", who isn't? I would say the US is just as much (if not more) asleep, wondering unwittingly into a techno-dystopian future. Or look at how they have been whiplashed by China's rise (which has their own big problems too). Not to mention the more recent disastrous reputational decline on the world scene. Of course, that wouldn't happen if Americans would recognize these dangers and not imagine themselves self-importantly as singularly awake at the wheel of international politics and economics while the car is heading head-long into the proverbial ditch.
*I assume that this is what you mean by "over-indexing" – I'm not feeling like digging around for the origin and exact meaning of this phrase, which is definitely not common English.
> I'm not feeling like digging around for the origin and exact meaning of this phrase, which is definitely not common English.
Just to start here, the term is very common in America.
> Declaring that this phenomenon is "at the bottom of Europe's list of problems" might sound like surprising and important, but it's just plain false.
Well I can think of 3 problems right now that are much more pressing for Europe:
Russia’s invasion of Ukraine and the broad inability of the EU to come to its aid in 2022.
China and hollowing out of European manufacturing. Germany in particular is shedding manufacturing jobs - common knowledge but happy to provide a few sources if this is a new development for you.
Migration crises and war in the Middle East that the EU is unable to address militarily or diplomatically.
Donald Trump saying a few mean words and initiating tariffs obviously start to fall in the EU problems rankings once we just start talking about the global geopolitical and strategic situation.
> As for Europe being "asleep", who isn't? I would say the US is just as much (if not more) asleep,
Both the US and China are leaving the rest of the world behind. As you noted, both have issues. The US started during the first Trump term to begin addressing posed by China’s continued economic and military challenges and continued through the Biden term via various legislation and policies, and now continues again during Trump’s 2nd term.
You seem to look at actions like arresting Maduro or attacking Iran from the perspective that such actions are harming international reputation or are the whims of, well pick whichever word you already used to describe him, but these are actions showing that the US is instead of “sleeping” actually very much awake and taking important strategic and necessary action. I can walk you through those as well if you’d like.
By "franksystems" do you mean those cathedrals which will outlast most buildings we're putting up today? We could go even farther back in history: how about the Pantheon, still the largest unreinforced concrete dome ever made?
We recognized socially that people who engineer things need to have professional ethics and carry liability for their projects.
The whole reason we have engineers is because we used to let anyone call themselves that and lo and behold a bunch of bridges collapsed under load and killed people. Who could’ve thought?!
Almost 2/3rds of the professionals in my family are licensed engineers and we take it extremely seriously.
Canada has only been actually licensing software engineers for a short time compared to other disciplines - but the industry as a whole is due for a reckoning.
Sure, and that's laudable, both for Canada and for your family. But most other countries don't have such a professional certification program, and since a ton of brilliant creators of software never had one, should we declare them fake engineers? People like John Carmack, Fabrice Bellard, Richard Hipp, Linus Torvalds... I think ultimately results speak for themselves. Surely there are plenty of middling licensed Canadian software engineers as well. Let's not police language like this.
It’s not about skill level it’s about accountability and ethics.
If a bug that Linus introduces into the kernel kills someone, is he held accountable? No, of course not the kernel is provided without warranty or any kind of guarantee of fitness.
He’s not an engineer.
He’s a programmer. A very very good one. But he’s not an engineer.
Why would taking the more open, minimalist, configurable and ultimately diligent route means you won't be working on anything else?? Not to mention that pi has other advantages over Claude and Codex, read up on it. Also, improvements to the agent itself will pay more dividends the earlier they are applied. The tone of this message is waaaay off.
> Why would taking the more open, minimalist, configurable and ultimately diligent route means you won't be working on anything else??
You're using the same finite pool of time and tokens. Why waste your time with the perfect gear instead of focusing on just getting really good at running? Just go run and when you've pushed the limits and the gear becomes the difference, then optimize the gear to get to the next level.
While you're busy trying to optimize your harness, others are just building and shipping with the magical software factory.
What are these "others" shipping, slopware? Agents are not a "magical software factory", they are a tool with a lot of limitations, but which can speed up development in a sustainable way, when used wisely. And that includes configuring it in a way that complements the other tools in our toolkit.
Everyone's waking up to this simple truth: vibe coding like there's not tomorrow accumulates conceptual and technical debt at a unsustainable rate. Then when the "magical factory" gets mired in its own mess, it's back to the drawing board. This is the also what the makers of pi have discovered, if you listen to their talks about how pi came about. I don't believe there are any justification for the assumptions you make about their approach, nor am I seeing you presenting any either. As it is, you take just feels peevish and unfair, to be honest.
A story to share: friend vibe coded absolute slop with Replit starting late 2024 (!!). Absolute trash code. Hacked multiple times because his login code exposed the full user list on the FE (!!!). Hacker found a way to exploit his account confirmation email because it was all front-end and sent an email to every customer telling them he was hacked. One time called me up in a panic asking why his web page was randomly refreshing (turns out, he was serving it in dev mode via Vite with HMR). It was mistake after mistake after mistake.
But he started to get customers. First a handful, then a dozen, then enough to get legal threats from other vendors, and this year, his first "enterprise" deal providing software in a space that was long dominated by a duopoly of legacy providers.
Guess what he did? Just rewrote it with the latest models and hired one engineer to ensure agents followed better practices. It's a legit business now built by a tiny team using a magical software factory to produce absolute trash code, but in shipping it, he found a market and customers willing to pay him for an alternative to the duopoly.
See, at the end of the day, it's cute that you have the perfectly tuned harness, but that also means whatever time you spent tuning your harness, reading up on Pi, spending tokens on your custom plugins -- all of that time and resources could have been used just building something useful.
People use Replit to build websites too, and some of them might scratch enough of a need to make money this way. So what? Is this what I should be mightily impressed with? That some random dude vibe coded some slopware which he was able to convince some random others to pay him for? I'm personally more interested and impressed by brilliant technical achievements, even if less monetizable, than some hustle or another in some industry niche which only ever attracted the interest of two legacy players. This is Hacker News, not Hustler News after all.
reply