The LLM should not be seeing the raw DOM in its context window, but a highly simplified and compact version of it.
In general LLMs perform worse both when the context is larger and also when the context is less information dense.
To achieve good performance, all input to the prompt must be made as compact and information dense as possible.
I built a similar tool as well, but for automating generation of E2E browser tests.
Further, you can have sub-LLMs help with compacting aspects of the context prior to handing it off to the main LLM. (Note: it's important that, by design, HTML selectors cannot be hallucinated)
Modern LLMs are absolutely capable of interpreting web pages proficiently if implemented well.
That being said, things like this Claude product seem to be fundamentally poorly designed from both a security and general approach perspective and I don't agree at all that prompt engineering is remotely the right way to remediate this.
There are so many companies pushing out junk products where the AI is just handling the wrong part of the loop and pulls in far too much context to perform well.
This is exactly it! We built a browser agent and got awesome results by designing the context in a simplified/compact version + using small/efficient LLMs - it's smooth.sh if you'd like to try
> The LLM should not be seeing the raw DOM in its context window, but a highly simplified and compact version of it.
Precisely! There is already something accessibility tree that Chromium rendering engine constructs which is a semantically meaningful version of the DOM.
Is this true for very wealthy people, or only for above average earners?
There are many examples of very wealthy people having lots of children. Children are still a significant investment for high earners, but at a certain wealth level it becomes inconsequential.
Quick google shows some support for this idea:
"There is a new emerging trend where better-off men and women are more likely to have children than less well-off men and women."
ergo, there is probably a level of financial support/wealth at which people start having more children. Or more simply, the point at which the personal benefits outweigh the personal costs.
It's similar to how China dominated manufacturing in prior decades.
They have massive amounts of low cost labor and, unfortunately, the US has pretty large walls up preventing mass in-migration of white collar workers.
H1B is capped and also more of a lottery than a points based system.
If the US allowed mass white collar immigration, wages would decline materially which would make our industries more competitive for the next generation of software.
Right now the system is geared around protectionism (intended or not) and wage inflation for US local workers.
The current market wages in software are far far above what a global equilibrium would be. Though myself and I'm sure most others here have benefited from it in the short term.
To be clear, established companies with an existing market are fine for now and can do well with high wages.
But the next generation of companies that are chasing smaller markets and margins, ones that require more elbow grease to out-compete are underserved.
e.g. the entire DeepSeek team was paid less than a few Meta engineers (with 7 figure comp each)
"The firm offers 14-month pay for various positions and the highest offer is for deep learning researchers for artificial general intelligence (AGI), with a monthly salary between 80,000 yuan ($10,983) and 110,000 yuan, which could mean an annual income of up to 1.54 million yuan, the report said."
Sad to see people still parroting the 4% rule when you can get "risk free" US Treasuries, today, paying more than that. Not to speak of the numerous, still conservative, investments paying far higher.
This isn't the 2010s era with ultra low fixed income yields. If you intend to retire early, please educate yourself on the state of the market
The 4% rule requires increasing that amount at the rate of inflation throughout retirement. 30 years in the case of the original studies. Even with an inflation rate of 2.5%, the required withdrawal will more than double after 30 years. The 4.625% you can lock into a 30-year Treasury would not be enough.
The 4% rule suggests selling 4% of your principal per year.
Buying a 4.625% yielding bond does not require selling principal at all.
But you're taking my point too literally. You can easily buy a 6 or 7% yielding bond that is lower risk than equities, but still paying far more in a more certain fashion
You can buy REITs today paying 6% that will grow rents around the rate of inflation (O is one example).
You can far surpass a 4% yield on cost in year 1, while also locking in inflation adjusted income growth today, very trivially, without selling one dollar of principal.
Unfortunately any thread on this is drowned out by uninformed people, or people who don't understand retirement is first and foremost about securing a stable (and non anxiety inducing) cash flow. Not growing wealth maximally.
There have been many times in history where the broad index has been flat for years. 2000 to 2010 is a famous more recent period. Would you be comfortable selling 4% a year into year 9, having watched your wealth decline materially over the last decade?
I guarantee you the retiree in bonds getting 5% and maintaining 100% of their principal is experiencing much less anxiety. And bonds give you the optionality to swap to equities in a down market like 2000 or 2008
For REIT example (O), the yield is 5.61% and you need to pay tax on top of that. Assuming 25% tax, you will net 4.2%. Current inflation is 2.9% [0], so assuming principal conservation, you can only spend 1.4%. Much worse than 3.5/4% on the market.. Am I missing something?
So if you withdraw 4% in year 1, and your underlying equities are flat for 10 years because the market moves sideways, as it did in 2000 to 2010, what percentage are you withdrawing in year 10? It's more than 4%
That just makes my example above even worse for a retiree.
Is it true or not that you can secure well above 4% in year 1 YoC in a relatively low-risk manner in today's market, and also secure it against inflation?
Yes, this is true.
So either you disagree with this premise and can back it up with a meaningful counterpoint, or you can agree and move on.
> The 4% rule suggests selling 4% of your principal per year.
No, it doesn't. You misunderstand the 4% rule. The 4% rule states you sell 4% of your principal in the first year, then inflation-adjust the 4% value every year. If the stock market rockets and your portfolio jumps 30%, you still only sell/spend 4%+inflation. If your portfolio drops 20%, you still sell/spend 4% of first year + inflation.
> There have been many times in history where the broad index has been flat for years. 2000 to 2010 is a famous more recent period. Would you be comfortable selling 4% a year into year 9, having watched your wealth decline materially over the last decade?
This is exactly why the 4% rule has the allocation be 60% equities, 40% bonds, rebalancing every year. Bonds have their place for reducing volatility.
> I guarantee you the retiree in bonds getting 5% and maintaining 100% of their principal is experiencing much less anxiety. And bonds give you the optionality to swap to equities in a down market like 2000 or 2008
If you're 100% bonds, what criteria would you ever use to get back into equities?
What does the 4% rule have to do with yields of treasuries? This is a 30-year time horizon that changes spending purely based on inflation figures. Yields in the market do not matter.
TIPS would work, but you're not getting 4% on TIPS. Though you can get 2.375% on 30-year TIPS, which isn't bad, just not enough to suffice for the 4% rule.
The 4% rule was proposed by Bill Bengen in the context of a 30-year retirement. Others have studied it in other contexts (including an infinite horizon) but 30 years is where the "safe withdrawal rate" literature started.
You can lock in cash flowing assets today yielding 6%+ with very low risk. Think REITs with conservative portfolios and strong balance sheets.
You can lock in today's treasury rates for 30 years by buying a 30Y treasury bond.
So, yes, they will last if you understand where to put your money. The options are extremely numerous and plentiful now in cash flowing assets, and you don't have to deal with the uncertainty of selling off principal in down markets to finance your retirement
6% after inflation? The problem with 30yr bonds is that it is effectively 2% assuming 2.5% inflation rate. Stock returns have been generally around 10% even after accounting for inflation. A good mix is critical and what is a good mix depends on the personal circumstances.
Are you trying to retire with stable cash flow, or grow your wealth?
For many, retirement is first and foremost about generating a stable income. A guaranteed lower rate of return is usually desirable over a speculative higher rate of return.
I seriously doubt Apple would move to Bing by default, even if there were some short term monetary gain. Using a subpar/cluttered search interface is so far off from their brand image.
I find it 10x more likely Apple would suddenly find the motivation to make their own search engine if forced to end their deal with Google.
(They say they wouldn't under any circumstance, but seems to be posturing to me)
Regardless, it's clear getting paid Billions to give people the default they'd choose anyway is a good deal for them.
I wonder who at Google negotiated this, because the terms seem very bad for them. They only make sense if the premise was to prevent Apple from starting a competitor
> I seriously doubt Apple would move to Bing by default, even if there were some short term monetary gain. Using a subpar/cluttered search interface is so far off from their brand image.
> I find it 10x more likely Apple would suddenly find the motivation to make their own search engine if forced to end their deal with Google.
I find it 10x more likely that in such case they would use white label Bing and do front-end on their own.
> I find it 10x more likely Apple would suddenly find the motivation to make their own search engine if forced to end their deal with Google.
I think that could only work if someone successfully makes the case internally that an Apple search engine built in to Safari (not as a first-class web app) would boost the Apple brand and/or Safari market share enough to justify it. Maybe even offer it as a subscription.
To monetize it with ads would go completely against their DNA -- ad revenue incentivizes companies to violate users' privacy and build a sub-optimal UX. So it would have to be either a subscription or a platform feature.
It is funny and a bit sad that many root for anticompetitive behavior by these companies out of what appears to be largely simply due to fanboyism.
Making markets highly competitive and open to new entrants/innovation is far better for society in the long run.
Is society better off if Visa can take 5% of every Transaction? Apple/Google 30%? Clearly not.
In a competitive market, margins will trend towards marginal value add of the player. Margins well in excess of the add are signs a market is not competitive.
Open protocols for payments, storefronts, Identity/Auth, messaging etc can solve most of these inefficiencies.
Eventually government will get smarter on technology. (Maybe on the cusp?)
It came out in the Epic trial that 90% of App Store revenue comes from pay to win games and loot boxes. Most of the other popular apps on either store are clients to services where Apple doesn’t get a cut at all.
You’ll have to forgive me if I don’t feel sorry for those types of apps - the 90%.
Well, seeing that I don’t buy pay to win in app purchases of games and none of my subscription services - Prime, various streaming services, Office365. etc - were purchased through the App Store, the other 10% don’t apply to me.
People pay Microsoft all of the time to buy games for XBox
It's not relevant. You're looking for something to complain about and latching on where it doesn't fit.
Even full approval of Apple's motion would not be rooting for anticompetitive behavior. It makes sense for Apple to be involved in this rulemaking. And it does seem like too much if the rule blocks any dealing between the two companies in any market.
But the above comment was far weaker than that. It was just saying a restriction that strong would prevent things that are not already illegal. That is a basic fact.
Induced Demand is a poorly conceived mental model.
The concept of "Induced Demand" is easily explained by the default state of the downward sloping demand curve, and upward sloping supply curve.
In basic economic theory, if you reduce the cost of a good, more people will consume it.
e.g. the classic building a highway example; there was always demand for cheap housing with accessibility to downtowns, but the supply of cheap housing with accessibility to downtowns did not exist prior to building the highway there. The "demand curve" doesn't shift at all, you're just moving along it as perception of cost changes.
Has there ever been a case where a highway was run through the middle of nowhere and traffic didn't increase? It's intuitive why
> The concept of "Induced Demand" is easily explained by the default state of the downward sloping demand curve, and upward sloping supply curve. In basic economic theory, if you reduce the cost of a good, more people will consume it.
It is easily explained by that because that is literally the definition of induced demand (see sentence one of wikipedia). The concept has a name because its important to discuss the externalities and long term implications of that additional consumption.
w.r.t your highway example, how you define the market is extremely important and is sensitive to context. The market for "transportation between suburb X and city Y" experiences a durable change in the demand curve as a result of the construction of all that cheap housing. Both market definitions are valid but if your concern is e.g. urban sprawl then contextually one is a lot more relevant than the other. All that said, you can think of the change in the demand curve of market B not as induced demand itself, but as a consequence of realized latent demand (i.e. induced demand) in market A (cheap housing with accessibility to downtown). Alternative solutions to realizing latent demand in Market A (public transit, denser housing, etc.) have different and potentially preferable externalities which is why considering induced demand and its consequences are important.
"Induced Demand" is often, incorrectly, talked about as a shift in the demand curve in response to increase in supply.
There is no shift. I am addressing this common misunderstanding, not debating the wording in the Wikipedia article.
"Induced Demand" is a misnomer, as the demand was always there at the given price. It is not induced, just realized. If I offer you a gold bar for $0, did I induce your demand to accept it?
Most people will always have demand for goods offered below their perceived intrinsic value.
Ultimately it's semantics around definitions, but the thinking of lay people around this concept is typically more of the shifting demand curve, not realization along the existing curve
>"Induced Demand" is a misnomer, as the demand was always there at the given price.
That's not necessarily true. Suppose you're the government and you produce food for free, and every year people eat everything you make, and everyone is well-fed. You decide you want to prepare for a famine, so this year you start more farms such that next year you make 20% more food. The first two months you're able to save, but when people see that there's more food available, they change their habits and start doing even more exercise than before, and so they eat more until they eat all the food every month again.
In general LLMs perform worse both when the context is larger and also when the context is less information dense.
To achieve good performance, all input to the prompt must be made as compact and information dense as possible.
I built a similar tool as well, but for automating generation of E2E browser tests.
Further, you can have sub-LLMs help with compacting aspects of the context prior to handing it off to the main LLM. (Note: it's important that, by design, HTML selectors cannot be hallucinated)
Modern LLMs are absolutely capable of interpreting web pages proficiently if implemented well.
That being said, things like this Claude product seem to be fundamentally poorly designed from both a security and general approach perspective and I don't agree at all that prompt engineering is remotely the right way to remediate this.
There are so many companies pushing out junk products where the AI is just handling the wrong part of the loop and pulls in far too much context to perform well.