The article is quite general. Here's some notes on how AI is being used to do AI research at frontier labs specifically. It's not the singularity (yet?) but it's heading in that direction.
Most training is now actually inference, not directly gradient descent. Reinforcement learning requires the generation of lots of 'rollouts' that are then compared with each other via an algorithm like GRPO. Or they might be compared using a critic model - AI judging AI and causing it to self improve. Generating a rollout means inference. And there's lots of data cleaning by older models. This has been called in the past 'textbook' or 'curriculum' learning, not sure what it's called now. But AI is also used for things like data/document labelling, transcription of videos, detection of images/videos with watermarks or subtitles, elimination of content that shouldn't be in the dataset, creation of new content that should and so on.
AI has proven capable of some routine work, like brute-force optimizing GPU kernels or doing hyperparameter sweeps.
Obviously, researchers are all using coding agents too.
So that's a few ways AI is self-improving. But there are lots of other ways in which even frontier models are still beaten by human researchers. Experiments in closing the loop have failed. For instance, people have tried giving the latest models access to some GPUs and an old version of an AI codebase that was recently optimized by human researchers (a NanoChat speed run goal, I believe). Could the models match the performance of the AI researchers? Nope. They only got 10% as far as the humans did, mostly because their approach was uninspired. They wasted a lot of time and budget doing low-IQ stuff like hyperparameter tuning. The humans had many other tactics like studying the research literature and inventing new algorithms that the models didn't even attempt.
The bottleneck is therefore currently the level of insight and inspiration the models are capable of. I've also seen this in my own work. I come up with an idea I think is novel and see if I can get a frontier model to reach the same idea. It never works without questions so leading it's more or less pointless.
It's very unclear why AI struggles so much with innovation yet can invent new songs, poems etc without apparent difficulty. Obvious answers like "it's not in the training set" don't feel right to me, the issue is deeper.
It's the Einstein question. Given an LLM trained on all written word up until right before Einstein's work, could the current SOTA in LLM architecture rediscover relativity? The jury is out on that, but until someone runs that experiment and proves otherwise, we have to assume LLMs simply aren't capable of that kind of brilliance. They're still impressive and useful and also stupid at times, but at the end of the day, no LLM has a gut to make a gut decision with. Unlike us.
I learned a lot about the AML system many years ago and still have some books about how it works on my bookshelf. It's fascinating stuff.
A few minor comments.
My wife and I have owned and used 1000 CHF notes quite a few times in the past. The last two times we moved apartments I paid part of the moving fee with a 1000 CHF note. We've also bought furniture this way. Nobody was surprised to see this and the notes were accepted without question or comment. To a person who spent their life in Britain this sounds absurd because the British government has - true to form - been trying to wipe cash out for many years to improve surveillance. You can't get any high value notes there, they just don't exist, because the state assumes that anything it can't see must automatically be suspicious. And there's so much street crime, and the police care so little about burglary, it would be very dangerous to hold such notes. But in Switzerland it's safe and the government doesn't try to wipe cash out, so paying with high value notes is common. (Although bank notes are in no way as private as people assume and can be tracked quite well, because they don't tend to circulate far.) This situation makes a mockery of the recommendation to fight crime by removing high value bank notes. The UK did this already and ML is out of control there: criminals just don't care.
I looked into the case of George Cottrell once. The case against Cottrell collapsed because it was founded on entrapment (the eight months was a plea deal in the usual American fashion, and doesn't mean much). It boiled down to undercover FBI agents asking Cottrell, "how could we launder money" and he explained how to do it, in the way anyone familiar with the topic could. He didn't make any offers to actually do it, didn't do it, and that's why "could be jailed for up to 20 years" turned into an eight month plea bargain to let prosecutors save face. Then after he was released he trolled US LE by writing a book called "How to launder money". Usually not given is the subtitle: "A guide for law enforcement and politicians". It's not written for criminals but people love to omit that detail.
Carousel fraud in the EU is a huge problem that governments hate to talk about because they don't know how to fix it and fear that by talking about it, they'll just teach more people how to do it. It's an infinite money glitch but in the real world. It's interesting that the UK thinks they solved it. I suspect they didn't, rather, enforcement collapsed elsewhere and it became easier to just go back to other ways of scamming the government.
> You can't get any high value notes there, they just don't exist
Yes, although this is mostly by capping the highest regularly circulating note at £50 after the war and then waiting for inflation.
> And there's so much street crime,
[mostly false]
> and the police care so little about burglary,
Sadly true (including "find my iPhone" reports; there was a joke during the Mandelson scandal that this was the one time the Met had managed to locate a phone)
> The UK did this already and ML is out of control there: criminals just don't care.
Yes, which makes a sort of orthogonal point about whether or not cash is actually important for this. There's the conspicuously suspicious businesses ("American sweet shops"), but also more complicated stuff going on (Scottish Limited Partnerships were in the news). Then there's all the Crown dependencies, which are a total financial wild west still.
> Carousel fraud in the EU is a huge problem that governments hate to talk about because they don't know how to fix it
God yes. This is a significant problem in VAT as a concept; I don't understand why the EU loves VAT so much.
I'll give you one better. I know someone who had their nearly new Range Rover stolen in Manchester - reported to the police etc. Few days later, they found it parked at a car park near a big supermarket. Rang the police, they said well, if you still have the keys...just take it? And he was like hang on, you don't want to look at it, check for drugs, take fingerprints, you know, do any actual police stuff stuff around stolen property? And they were like nope, don't have the time or the people to come out, if you have the keys just take the car back and make sure you tell your insurer you got it, that's all we can do.
I've heard that this is quite common. The criminals drive the car a short distance and leave it for a few days to check if it has a hidden tracker. If the vehicle isn't recovered they assume it is clean.
The story I heard is the car is stolen and used briefly to commit other crimes like robberies or general mischief. Then the car is discarded leaving the surveillance trail largely dead or at least difficult to follow.
Depends on the vehicle. Range Rovers are commonly stolen to be exported for countries where registering them is not a problem and where buyers are likely to pay cash. Some cars are stolen to be broken up for parts, some are used for other crimes, and some are exported and re-sold as-is. LR products are very frequently in that last group.
"I don't understand why the EU loves VAT so much."
Because it's easy money as taxation goes. Facing growing fiscal deficit and worsening credit score, the first thing the government in Romania did last summer was to rise the (general) VAT quota and cut on some VAT exceptions. It works quicker and more reliably than other means for securing the budget needs.
The VAT related fraudulent schemes are a problem in EU as many other things are, but they are investigated, often prosecuted, and written about. For anyone interested, more can be found at the European Public Prosecutor's Office's site: https://www.eppo.europa.eu/en/media/news
VAT/GST is a direct tax on productive economic activity (as opposed to some proxy like income or capital gains) and I'm sure there are good economic arguments why that's the best kind of tax.
I'm not suprised that in the swiss economy no one bats an eye at 1000 CHF bank notes. After all the swiss are historically known for being the classy alternative to launder and store your ill gotten gains from, for example, your stint as the dictator of an African country.
But there has been some changes in recent years so I don't know how it is today.
The absolute peak of that was, of course, during WW2 when vast amounts of stolen or looted money, gold, and other valuables ended up being laundered through Switzerland and kept in anonymous accounts. Mostly by Nazis, although not exclusively. There was a long campaign of litigation by the descendants of Holocaust victims to get some of it back.
In the 21st century the US eventually pressured them into not being a tax haven for anonymous money hidden by US nationals.
The twin questions of tax and terrorism remain as pressure against money laundering.
The usual (gu)estimate is that 1 to 3% of UK GDP is exposed to suspicious foreign wealth. It is estimated to be 15 to 20% of Switzerland GDP today.
So indeed, it doesn't make sense for Switzerland to limit circulation of cash or increase tracability.
Those estimates aren't based on anything real. The entire financial system including things like insurance and pensions is only 10% of Swiss GDP. Private finance is like 1% or less. And the AML rules are the same as everywhere else because they're standardized by the FATF.
You get a lot of nonsensical talk in other countries about the Swiss economy because the alternative would be to admit that it's a genuinely strong economy and thus that the Swiss are doing things right. There's a culture in the British civil service of assuming there's nothing that can be learned from other countries policy-wise.
That's a sort of standard cope found abroad but bears no relation to reality. It's not easier to launder money in Switzerland vs anywhere else. Actually it's much harder because of the generally high level of policing here. Dictators and oligarchs often end up in London instead.
Look at the article and note the common themes: money is frequently laundered through dodgy chains of high street stores that don't seem to have any actual customers. Everyone is aware of this problem but nobody solves it. What the article doesn't explain is why: it's because these chains are largely owned by ethnic minorities and the left wing governments that have ruled Britain for decades are terrified of anything that looks like an ethnically targeted crackdown, or anything that could be called "Islamophobia".
Good luck running a fake vape shop in a Swiss village, lol. I've never seen or heard of such a thing here. In Britain you can just drive for a while and see lots of them. Look at the sibling comments where British police won't even investigate car theft (the idea there's not lots of street crime there is crazy). Come visit Zürich and it won't take long until you see bobbies on the beat, just cruising around looking for trouble. Levels of attention to low level street crime are completely different.
The origins of the Switzerland/ML link are the numbered (anonymous) bank accounts available and used in WW2, often by Jews trying to preserve their wealth from confiscation by the Nazis. Along with a general culture of financial privacy in which the local governments are constrained by law from obtaining all your transaction data from banks.
The AML system was invented by Americans who cared a lot about the war on drugs, tax evasion, and later, the war on terrorism. They didn't care about financial privacy at all, probably because they never lived right next to Nazis or communists, or experience floods of refugees fleeing totalitarian governments during WW2 (not on the scale Switzerland did). So the global AML system has been built largely by America threatening huge trade sanctions on any country that didn't agree to comply and financial privacy be damned. After all, Uncle Sam is the very avatar of democracy and freedom so who could object to it knowing everything?
Culture of financial privacy <-> American sanctions. Who wins? The answer is the US wins and Switzerland implemented all the rules long ago. Swiss citizens get some level of privacy from their own government, but Swiss banks will hand over all your data and then some to the US government if you're a "US person" - a category much broader than merely being a US citizen. So banks and sometimes other companies here often have you declare that you're not a US person because dealing with such persons results in the full force of the AML/FATCA compliance cannon being pointed at the org and it's too expensive and dangerous to deal with.
> The UK did this already and ML is out of control there: criminals just don't care.
From 18th May:
“A new £30m High Street organised crime unit has been announced by the government after the BBC's year-long investigative reporting into illegal mini-marts, vape shops and barbers.”
I don't know of one that covers everything. Treasury's War is a sort of auto-biographical insider oriented history written from the perspective of someone in the Treasury Department, using the system to chase "terrorists" (often just random Muslim NGOs or orgs with zero evidence of terrorist involvement). Other books cover parts of the system incidentally, like the Banker's Plumbers Handbook (warning: self published, roughly edited).
A lot of what I learned about it, I learned by reading random documents published by governments designed for people working in regulated industries. Or by talking to people who worked in banks or regulators themselves. AML doesn't just cover banks but also in some countries casinos, car dealerships, lawyers, auction houses... it's like trying to push down on mercury so the scope of the rules constantly expands.
>To a person who spent their life in Britain this sounds absurd because the British government has - true to form - been trying to wipe cash out for many years to improve surveillance.
Oh come of Facebook for christ sake. Absolute hogwash.
Calling your students misogynists is a shamefully harsh attack on them without any evidence to back it up. That feedback is exactly what a male teacher would get if he had the same career history as you.
You say in a post below that your total games industry experience was a single internship at Blizzard and then a second stint where you "quickly realized" you didn't want to be a games dev at all, and went back into academia where you have been ever since. You say you made a game as part of your PhD, but it's actually a speech therapy program you describe as research. There's nothing wrong with that project for what it is, but your students aren't criticizing you because you're a woman, they're saying they wanted a teacher who spent time in the games industry making the sort of games they themselves would play.
I wouldn't bother pointing this contradiction out normally, but it's just so socially destructive to ask students for feedback and then attack them with the nastiest accusation you have access to, just because they requested a more experienced teacher. Poor kids! It's this kind of thing that results in recommendations to just avoid university entirely. Why sign up for being abused by a teacher like that?
> You say in a post below that your total games industry experience was a single internship at Blizzard and then a second stint where you "quickly realized" you didn't want to be a games dev at all
That is not my entire games experience. I have 15 years total, spanning Game Master, lead gameplay engineer, game engineering director, and CTO. I was asked my route to academia, not my entire Gaming Industry CV.
> they're saying they wanted a teacher who spent time in the games industry making the sort of games they themselves would play.
Nearly all of them have played games I've worked on, and can even find my name in the credits.
> Calling your students misogynists is a shamefully harsh attack on them without any evidence to back it up
You're going to make a bunch of assumptions based on a summary of my academic career and then try to insist that misogyny doesn't exist in tech?
Yes, I've never seen any misogyny in tech, if anything it's the other way around where men are told there are too many of them and their employer would prefer more women. But my criticism of your post isn't related to you being a woman, it's to do with you immediately leaping to the worst possible interpretation of your student feedback. Why assume bad faith immediately? Where's the evidence to support that? Do you just assume any negative feedback always has a hidden agenda?
Look - you say you have made games, and that your students have all played games where they could find your name in the credits. You also get feedback that your students don't believe you. This is a weird problem to have and should be trivial to fix if true. Just... boot up the games and show them where you're credited as the lead gameplay engineer? What do they say when you do that?
That catch-22 is supposed to be broken by the bank. It's a two phase commit where you open the account in a special state where you can only deposit the capital. Then the bank gives you evidence you've done so, you take that to the notary and open the company, then send the evidence you've done that back to the bank to convert it into a full account.
It's a bizarre system that Switzerland uses too. I've done it twice. Unfortunately the German speaking world has a lot of rules that are trying to eliminate all risk for investors and employees. The GmbH/AG capital requirements are just the start.
The next fun thing you might have encountered, at least in Switzerland, are rules that literally say your company's assets can't fall below 50% of your initial capitalization. If it does you're supposed to raise funds or make more investment of your own private capital and this rule pierces the usual liability requirements. Even more fun: it turns out that this law isn't actually enforced and locals regularly ignore it. But bad accountants won't tell you that. They'll just inform you of the law when you do your yearly accounts.
Then you have wealth taxes that cover the valuation of a startup as if it were a cash position. So if you raise $100M in investor funding then whatever shares you have left over are considered to be liquid assets you can offload at will, and are wealth taxed as such. The fact that the shares don't trade in a liquid market is irrelevant to the tax authorities. In Zürich at least that got patched by the local tax office deciding that startup shares aren't counted for the wealth tax, but this just means you have to be able to convince the tax authority that your company is a startup. The way they determine this is more or less just the opinion of whoever at the tax office assesses your case. Does it sound "startuppy" enough?
Fixing this stuff isn't hard, but it never gets fixed because European politics is both quite stagnant and dominated by people who view hostility to business as a virtue signal. They don't want to fix it because they think businesses are sort of like oil fields. They just exist, lying around naturally, and the only question is how to maximally exploit them.
re: catch-22. I was surprised to learn of it, but then even more surprised to get caught up in an endless loop even when I followed the process.
For European countries it seems like a left-hand-right-hand problem: I go to a party at the Los Angeles Austrian consulate, or a "Start Up Austria" event in SF and listen to them pitch bringing businesses to Austria (the left hand). But back home the "right hand" and culture seems to look down at people who have accumulated wealth, and be hostile to the idea of the most simple reforms that would actually make someone want to move their business to Austria/EU. For example, the intrenched interests of needing a notary paid a few thousand Euros, are screwing the whole country so one set of people can fill an archaic role.
Do you know how much a notary costs in the US? If you go to a UPS or FedEx store, it is typically about $10-15. If you go to your own bank they do it for free.
And get this, when you incorporate a company in a US state, guess how many things need to be notarized? None. As you fill out the forms you are swearing to their truth, and accepting a notice that you are committing perjury if you are untruthful. Do they check your statements right then to make sure you're not lying - absolutely not! Why bother, you haven't started anything yet. But by God, if 5 years later when you're making money, the IRS finds out that you lied, then you're going to wish you were living in another country.
In other words, the timeframe for the risk concern is completely backwards in Europe: the risk management basically stops 95% of things from happening in the first place, including the 0.001% the might be fraudulent. Wouldn't it be better to let 1,000,000+ companies get formed, then 50,000 of them naturally become a meaningful success, then take a closer look at their compliance once they are truly a real business.
I had some ignorant biases against doing business in Europe before starting this process - and I was hoping that experience would change my mind - but the funny thing is that my biases weren't strong enough! (my bank and incorporating story above is just one example of many). No thank you EU!
(My first businesses were started in Canada, and I thought Canada was so backwards compared to the US in being business friendly, but now I realize Canada is maybe 90% as business friendly as the USA. EU, in my experience is like 5% as business friendly as USA)
> The Pope is pleading for multilateralism and responsible regulation of technology.
According to the Economist at least, he doesn't seem to know what he wants. The encyclical sounds like a grabbag of every progressive meme and worry out there, whether they contradict each other or not.
You can't have both multilateralism and AI regulation (however that's defined). If you have genuine multilateralism then there will always be some jurisdictions that say they don't want to regulate and gain a competitive advantage by doing so. Because AI is symbolic and accessed over networks, in a truly multilateral world there is no such thing as AI regulation, really. Model development and serving will slowly migrate to jurisdictions that don't pin it down too much.
The only way to stop this is for every jurisdiction in the world to agree on the same set of rules. Which is the One World Government solution, normally in the 21st century approximated with economic pressure e.g. threatening to sanction or blacklist your country if you don't comply with some new rules. The anti-money laundering system is an example of that. And if you become familiar with the stories of its abuse, then AML can sound pretty darn Antichristy. So Thiel isn't far off.
Describing the pope’s proposals as progressive and anti-money laundering laws as the antichrist… this is like a parody of the most blinders-on kind of libertarianism.
For those of you playing at home, you can definitely have multi-lateral agreements without creating a one world government. We’ve had a chemical weapons ban for decades over which many of the multi-lateral parties were in hot and cold wars with each other. The nations are not going to magically combine over the presence of a treaty. Not how power works.
Anytime the pope comes up on HN it’s always a really quick way to find out who is very happy to step away too far out on a limb as they discuss subjects they know very little about (to be clear not talking about you)
There are plenty of rules that we apply across the board. No nuclear weapons for anyone who's not already got them is an example. This doesn't take some spooky one world government to do. This post is wild. Essentially, you're saying that any attempt to regulate AI as the existential threat that it almost certainly is the antichrist. It's bonkers.
That specific rule is enforced by assassinating the leaders and scientists of governments that don't agree to it. See: Iran. I don't think that's what anyone means when they say multilateralism. It's effectively an ad hoc global government defined by the reach of air power.
Then I guess all of human history is one big ad hoc global government because we've enforced the thing we want at the business end of a spear since there have been people.
Other way around. Nothing in history has ever been enforced "across the board", including the rule against developing nukes. A bunch of countries managed to develop them, others were stopped (so far). It wasn't done via "multilateralism" but rather by killing people who were doing it.
Unless your vision for AI regulation involves drone striking offices where AI researchers work, then there's no way to enforce it globally.
I'm not sure what point you're making anymore. The goalposts are floating in the sea. Yes, we could blow up facilities with advanced AI infrastructure. It wouldn't even be hard to locate.
I think you're not reading it in the spirit it's intended. There's a section towards the end (Chapter 5, I think) that is full of policy prescriptions. But most of the encyclical isn't "about" AI, it's "about" Catholicism, and is using AI as a lens to talk about principles the church has been building up over a century. In that sense the document is less concerned with frontier models and disinformation than it is with establishing Catholic social doctrine --- subsidiarity, solidarity, the common good, etc.
As far as the church is concerned, AI as an issue will come and go, but the ordering and prioritization of human relationships is timeless, and is the important issue. The subtext of the whole thing is that if you get the principles right, the tech policy will fall into place.
You can argue with those principles, but at that point you really are just arguing with Catholicism itself, which is fine, but is besides the point.
(I'm not engaging with or disputing your takes on policy, only with your comment as a critique of the encyclical itself.)
Perhaps, but I think that's a bit generous. Let's look at Chapter 3, titled "TECHNOLOGY AND DOMINANCE. THE GRANDEUR OF HUMANITY IN LIGHT OF THE PROMISES OF AI."
This whole section is clearly about AI and social policy. It makes occasional Biblical references but if you strip those out it sounds like any Democrat podcast. If random people were given these quotes stripped of context, how many would guess it was the Pope?
For example: > What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating
That's a demand for AI regulation.
Then take the paragraph that starts with:
> In many cases within the digital context, control over platforms, infrastructure, data and computing power does not rest with States, but with major economic and technological actors [snip]
The whole paragraph has nothing to do with Catholicism. It could have been written by the EU Commission and you'd never know. In it he appears to argue for the nationalization of AI labs, using standard progressivist claims.
Later the Pope argues once again for the nationalization of not just AI labs but all intellectual property held by the computing industry, using an argument I'm not afraid to condemn as theologically specious. In "The principle of the universal destination of goods" he says first that things like earth and water are given by God and thus everyone has a "right" to use them as they wish. From a theological perspective this is reasonable, albeit not from an economic perspective. But then he argues that patents, algorithms, datacenters and digital platforms are exactly the same as soil and water, and thus everyone should have them for free. That's nonsense. The religious justification for the first is that God made planet Earth, but He obviously didn't invent the transformer algorithm so why would the same logic apply?
All this is just standard left wing politics. The only theological justification I could find in the first part of this chapter is that some other recent Pope agrees with him.
I don't have any problem with Catholics or Catholicism. In fact I've written a whole essay arguing that AI raises issues only religion can deal with:
Religion has something to contribute when it comes to pondering questions like, what is AI? Does it deserve compassion and feelings, does it have consciousness and free will, or is it just a machine? Does the creation of it make us challengers to God or would He have approved of us making creatures in our own image? But the Pope doesn't engage with those topics. Instead we get advocacy for government power. The world has enough of such politics already.
> if you strip those out it sounds like any Democrat podcast
The fact that the bloody Pope sounds like what appears to be a left-wing party in the US' Overton window should be a big kick in the arse. In most of the rest of the west, these are classical conservative values, and indeed more aligned with the gospel than anything coming from the Republican Party these days. As a leftist, I find Magnifica Humanitas to be interesting, because it's a view point that is rooted in a rich history and profound thinking. But that's not a socialist doctrine at all. Its situation is very close to Rerum Novarum: it was more social than what the capitalist magnates wanted, but it was really far from things like communism or revolutionary socialism. Leo XIV does not hide his admiration for Leo XIII, and he sees many similarities with the state of the society they live in. On that, I think I agree with him.
So, I'm not sure about your religious background but conversing about this will get tedious very quickly if I have to hedge everything I'm saying, so from the jump let me just say I respect your writing on HN and I assume you're not Catholic, and you can correct me if I'm making any broken assumptions about you.
Also, just to get this out of the way: I said "Chapter 5" was the section full of policy prescriptions, but that was from memory, and, as you've noted, it's "Chapter 3". I agree there's a run in this where he gets pretty prescriptive! But I still think "policy document" is the wrong way to ready this.
Leo goes way out of his way, in the tradition of all Catholic popes over the last century, to ground what he's saying in a long through-line of doctrine. So in both your quotes, about the need for regulation: it's not really about policy.
I think Leo is first espousing a normie view (neither especially "left" nor especially libertarian) about regulation and risk, and then using it as an object lesson about the Catholic principle of Participation. Catholicism is big on ordering and prioritizing relationships between humans. We are supposed to be making decisions together for the Common Good, and we are supposed to recognize that decision-making happens (must happen) at different levels, from the state to local communities to families (this is Subsidiarity).
I flinched at the intellectual property bit too. But the point he's making in context is clear; it's Catholic and Christian doctrine going all the way back to Genesis. The literal named principle "Universal destination of goods" goes back to Vatican 2. The "codification" of these principles happened under Benedict XVI, not my favorite pope (I'm much more to the left than you are) and obviously no squish.
I think you're reading too much American politics into this, for what it's worth. Leo XIV took his name from Leo XIII, who in the late 19th century wrote Rerum Novarum, which was was in part a reaction to Marxist/Socialist thought and totalizing class conflict, recognizes the importance of worker welfare and the dignity of labor, but very specifically does not reject private property (private property is a necessary precondition for the agency of the family unit, which is central to Catholic doctrine). If we dig in I think we'll quickly find a lot of ideas that a doctrinaire leftist would recoil from!
But my big point is that people are all excited to read the Vatican's AI policy document, and the Vatican is uninterested in publishing AI policy; what it wants to do is continue to indoctrinate Catholics on the core tenets of Catholic social doctrine: Subsidiarity, Solidarity, The Common Good, Human Dignity.
Thanks. I respect Christianity and feel it has a lot to contribute to the topic of AI, even if not like this, but I'm not a Catholic.
I haven't read the other chapters so will take you at your word about what they say and the Christian agenda in them.
It's tempting to describe the sort of politics in chapter three as US politics because that's roughly where it kicked off first, but there's nothing US specific about it. Leftism has always been a very globally consistent set of beliefs. We wouldn't describe communism as British politics, for example, even though Marx developed his beliefs while living in London and wrote in a British context (Das Kapital is full of references to Parliament and living conditions in 19th century Britain).
What I'd really like to see from the Vatican is engagement with the question of consciousness, and why AI should or should not be considered something with rights. I think a lot of people view this question as obvious but when I studied it, I ended feeling that it's neither obvious nor something that can be answered from first principles. A bit like animal rights, a principled answer essentially requires some kind of religious grounding.
In an infinitely-large "truly multilateral world", what you are saying is true under the assumption that unregulated AI provides a competitive advantage for the jurisdiction, assuming preferences are each sampled from a totally-supported probability distribution. But we only have finitely-many jurisdictions, and it's not clear that AI accelerationism is actually good for anyone (except those extracting wealth from the corresponding financial bubble), so this conclusion doesn't follow.
I mean yes, if we twist meanings of words enough, Pope is progressive. Except he is not, he is conservative catholic pushing for old school conservative catholic doctrine. He is not far right, he is not prosperity gosphel guy, but catholic doctrine was never that.
Of course you can have multilateralism and regulations. And no, AML is not an antichrist.
And Thiel with his plan yo create totalitatian fascist word is one of the greater danger to most of us. Way greater then AML regulations.
Plenty of people besides the Pope have made essentially the same criticisms about AI and the dangers of the cultlike influence it seems to have on our society. Every one of those people were dismissed as paranoid and ignorant Luddites who simply feared and hated progress if not humanity itself until the Pope came along and voiced an opinion on the matter. Then and only then was criticism of AI taken to be valid, or at least taken seriously, because of the invocation of religious authority. (but now all the tradcaths are sedevacantists because the Pope is "woke" and obviously a Luddite who hates progress and humanity.)
I'll take it, but I really wish we didn't need it.
"Antichrist" is not really a serious word you can pin down, but the AML system is regularly used in ways that are very un-Christ-like. For instance, Christ said to love thy neighbour, to be the good Samaritan, that people should not be punished for the sins of their family members and he preached tolerance.
Now consider the unlucky German-Turkish journalist Hüseyin Dogru, who was recently placed under trade sanctions by the EU Commission due to his reporting. But they did it while he was living in Europe. The sanctions force everyone - including his neighbours and supermarkets - to refuse to sell him anything.
Then they realized, what if his wife buys him food? So they sanctioned his wife too.
Then they realized, what if his parents buy him food? So they sanctioned his parents as well.
Literally the entire family has been put to economic death. The state will imprison anyone who helps them and confiscate the entire net worth of anyone who conceivably might help them. All appeals have been denied.
They are now completely screwed and reduced to living on a subsidence budget of ~500 EUR/month from the government, calculated to be just enough that his family don't literally die of starvation.
This is possible because of the systems-level implementation of the AML/sanctions system and its existence outside any kind of justice system. It's the kind of thing that Thiel meant by a totalitarian antichrist. The Bible warned of the "mark of the beast" on people's hands that would prevent people buying and selling if the antichrist doesn't approve of him. Well, a revoked EMV contactless credit card is basically that. If Jesus were alive today he would presumably have harsh criticisms of this kind of thing.
Not really. Search engines are a tech so centralized only two of them exist in the west, Google and Bing. There are zero open source search engines of any usable quality. Whereas there are lots of models out there, some free to download.
"only two search engines exist in the west" and "only two search engines in the west are of usable quality to me" are contradictory statements.
The models free to download aren't the models used by OpenAI, Anthropic and Google. You aren't going to get all of OpenAI downloaded to your desktop and running fully on just your hardware.
And in each case (search and AI) the potential to decentralize and maintain "usable quality" is limited by these technologies requiring physical infrastructure at a scale that isn't available to the home consumer.
I mean, they are the models used by Google, at least. Gemma is used by Google and you can download it freely, weights and all. OpenAI has released an open weights model although I don't know if they use it themselves.
They aren't as good as the full fat models but they're plenty useful for many real world tasks. Show me the open source web search engine that I can run locally and that's plenty useful for many real world tasks!
Most training is now actually inference, not directly gradient descent. Reinforcement learning requires the generation of lots of 'rollouts' that are then compared with each other via an algorithm like GRPO. Or they might be compared using a critic model - AI judging AI and causing it to self improve. Generating a rollout means inference. And there's lots of data cleaning by older models. This has been called in the past 'textbook' or 'curriculum' learning, not sure what it's called now. But AI is also used for things like data/document labelling, transcription of videos, detection of images/videos with watermarks or subtitles, elimination of content that shouldn't be in the dataset, creation of new content that should and so on.
AI has proven capable of some routine work, like brute-force optimizing GPU kernels or doing hyperparameter sweeps.
Obviously, researchers are all using coding agents too.
So that's a few ways AI is self-improving. But there are lots of other ways in which even frontier models are still beaten by human researchers. Experiments in closing the loop have failed. For instance, people have tried giving the latest models access to some GPUs and an old version of an AI codebase that was recently optimized by human researchers (a NanoChat speed run goal, I believe). Could the models match the performance of the AI researchers? Nope. They only got 10% as far as the humans did, mostly because their approach was uninspired. They wasted a lot of time and budget doing low-IQ stuff like hyperparameter tuning. The humans had many other tactics like studying the research literature and inventing new algorithms that the models didn't even attempt.
The bottleneck is therefore currently the level of insight and inspiration the models are capable of. I've also seen this in my own work. I come up with an idea I think is novel and see if I can get a frontier model to reach the same idea. It never works without questions so leading it's more or less pointless.
It's very unclear why AI struggles so much with innovation yet can invent new songs, poems etc without apparent difficulty. Obvious answers like "it's not in the training set" don't feel right to me, the issue is deeper.
reply