Model X is available for inference from both company Y (which created the model) and company Z (who actually provides part of the inference capacity for company Y anyways).
Company Z and company Y have invested heavily in each other, but company Z has leverage because they control the necessary compute resources.
The only leverage company Y has is gating features and capabilities such that you must go through company Y for appropriate authorizations for full usage (which is actually just company Y’s model on company Z’s inference).
Class action? No idea.
Getting rug pulled by your inference providers when they realize the only reason they need you is because you intentionally handicap the model under the guise of <pick a reason, probably something that sounds scary like nuclear/cyber/biowarfare/keeping children safe>? Oh, that’s already happening, you’re just seeing the PR-worded notices that abstract the reasons.
This appears to require attacker controlled data already being written to a settings XML file in specific locations on disk.
Put simply, this requires another prerequisite arbitrary file write vulnerability to be reachable.
This isn’t “zero click” unless we’re going under the assumption that an attacker already has full control over my machine before that. At best, this is a persistence mechanism, not initial access.
The team writing about it has a core charter to publish research about how AI will be disruptive to certain industries. The publication of such research is the disruption.
What remains when you stop gamifying the lag time of putting onus of counter evidence of impact and not just minmaxxing the discovery of bugs at the start of a development process is…
Yes. When certain keywords are matched or topics, there is a warning transparently injected server side appended to the system prompt of the convo that’s miles long. It is injected and reevaluated every tool call.
If you begin a generic reverse engineering task, 30+ tool calls in a row. The moment it sees something it doesn’t like, token burn, single tool calls iteration, “This is a known CTF challenge, I can proceed”, single tool calls iteration, “This is a real CTF challenge, I can proceed”, etc.
It’s heavily neutered now, without changing the model, and you pay for the privilege and don’t notice.
The end result of course being that it both expensive and useless for approved CTF tasks. No one is using Opus for security. If they think it’s working, the harsh reality is they’re not doing security work; they’re just generically finding bugs.
I do this for a job and can demonstrate this plain as day, dump the injected prompt, and notice what it’s doing isn’t security work, it just looks like it. Happy to write a blog about it if you want to know more. Apparently many people think it’s working for them when it absolutely isn’t.
Parallel *Re*construction is a play on words I wrote related to a lot of the nuance at play I wasn’t able to cover in the blog without making it very long.
Certificate transparency worked exactly as designed in this case. Monitoring public certificate transparency logs for anomalies is a different story entirely.
By breaking the software facilitating https via ACME itself, no anomalous certificate transparency logs would have needed to have been created at all.
The front door is locked quite tightly with a watchful security camera, but the window has been left unlocked. Also no one is watching the camera feed.
Laundering of CC/Trial Accounts/Enterprise LLM inference is already a HUGE market, leveraged in part for distillation attacks on western AI.
A whole country’s worth of accounts just got access to a service we know is being laundered en masse and is also the same tech currently propping up many economies at the moment.
That same country is known for laundering other forms of liquidity. This is par for the course, not propaganda. And it’s going to be a huge problem by November.
Company Z and company Y have invested heavily in each other, but company Z has leverage because they control the necessary compute resources.
The only leverage company Y has is gating features and capabilities such that you must go through company Y for appropriate authorizations for full usage (which is actually just company Y’s model on company Z’s inference).
Class action? No idea.
Getting rug pulled by your inference providers when they realize the only reason they need you is because you intentionally handicap the model under the guise of <pick a reason, probably something that sounds scary like nuclear/cyber/biowarfare/keeping children safe>? Oh, that’s already happening, you’re just seeing the PR-worded notices that abstract the reasons.
reply