For context, Google initially refused to merge JpegXL as a strategy play to promote AVIF, which was in use by other teams (i think Photos?). Internally, chrome engineers were supportive of jxl but were overridden by leadership.
I guess today’s post represents a change.
I don’t have any public evidence to support my claim, sorry. Take it or leave it
Between this, Iron Lung, and The Amazing Digital Circus finale getting a cinema release, I think this is shaping up to be a great year for small movie productions
The Amazing Digital Circus finale got completely shat on by fans after they leaked the ending and harassed Gooseworx because they didn't like it.
Is it worth being involved in indie production when success just guarantees a hate campaign against you? It will probably happen over Backrooms soon enough.
That’s an amazingly pessimistic take, and a shortsighted one. Glitch and the TADC team are more than just one person. Gooseworx is burned out, but her work helped kickstart many early careers among her staff. I think the only people who characterize TADC as a failure are folks who consume too much social media.
Like most projects, some indie productions (like TADC) end in creator burnout, others (like iron lung) end in moderate success, most probably end somewhere in between. We saw this with undertale, lights out, hazbin hotel, primer, el mariachi…
Real-world systems need to be able to say "I don't know." This is a test about misinformation after all, and overconfident responses contribute to that.
Teasing out the difference between "avoid" and "unknown" could be a different research question
TL;DR Ed argues that the deal between Anthropic and xAI could have been negotiated in such a way as to make Anthropic only appear profitable during its “ramp-up” period in June, which incidentally is also the month that Anthropic is making tons of other pricing changes.
Speculative decoding shouldn't actually change the accuracy of the response. The draft model drafts a couple tokens, and the inference framework verifies that the larger model would have picked them.
However, I've found that speculative decoders don't help much if you're running a model locally on limited hardware (for instance, my 32GB VRAM M1 Max from 2021). For one, you have to fit both the large and the small drafter model in memory. For another, if you're running a quantized model, the activation distribution is different enough that the draft model has a hard time guessing what's coming next.
My take is that speculative decoding is most useful on _very expensive_ prosumer/hobbyist setups where you have 128GB of VRAM and are running your local models with full fidelity. It's also helpful for inference providers where they can send output tokens at a computational cost slightly higher than their input token cost.
Your experience might be a bit dated, depending on when was the last time you tried it. MTP (which is a flavor of spec decoding) is showing really solid improvements on local models, even on consumer hardware.
In fact, as the article mentions, you get the biggest gains at low concurrency (so local should apply), with diminishing returns for higher concurrency (if you think in terms of unit of compute, it's probably better to serve more requests in parallel and get more throughput that way).
Eagle3 was great at low context tho, and this seems to improve things at high context. That's really cool, and hopefully it'll turn oout to be useful at those lengths. Eagle3 is also training dependant, so you could try training your own, if your use-cases diverge enough that 3rd party "generalist" models don't suit your needs. (in general nvda, redhat, etc. have provided general eagle3 models for popular families).
Agree. At high concurrency, you are better off spending the compute budget on parallel requests rather than draft prediction. The challenging part is that most deployment don't have static traffic profiles. A configuration that was right at launch may no longer be correct months later, and there is no signal that tells you when you have crossed the threshold.
The reason speculative decoding shows diminishing returns in batched workloads is because the principle of both is the same.
Speculative decoding predicts a group of tokens and verifies this group using the main model in one pass instead of decoding each token separately. Eg. for this group, the weights are loaded from RAM per group instead of per token: roughly the same computation is performed but not the same memory movement (and other overhead like kernel launches).
Batching utilizes the same mechanism, so speculative decoding is essentially an attempt to batch a single stream using prediction. An attempt, because the verification may reject some tokens if the prediction was inaccurate.
Thanks, appreciate the info. For whatever it’s worth regarding recency, I’m testing the main llama-cpp branch that was pulled and built on 2026-05-25 running unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Q4_K_M, my hardware platform is M1 Max 32GB VRAM. Is there a different fork or quant I should be using?
When Claude acts up, my strategy is to rewind the conversation to the point where the misunderstanding started, revise what I said just before, and then continue from there. I’ve found that letting one mistake enter the conversation seems to make further mistakes more likely.
The user already has omnipotent power over the agent’s sense of time and memory. We can rewrite what Claude sees and hears. Can’t do that with humans, but rewinding such an important function in Claude that it has a top-level keystroke.
Why spend time, tokens, and cortisol arguing and demanding the pet rock step through an apology protocol?
reply