FWIW I've only ever been on the API based plan at work and we never seem to run into the majority of the problems people seem to be very vocal about. Outages still affect us, and we do have the intermittent voodoo feeling of "Claude seems stupider today", but nothing persistent.
Of course it's a stupid amount of money sometimes, but I generally feel like we get what we're paying for.
We use them extensively in our agent framework at work for all sort of things. You can make up whatever you want, if the tags are semantic enough it just gets it, or you can add a bit of explanation about it in the system prompt or whatever.
- Circuit breakers when it seem like it's stuck in a loop
- Warnings about running low on context
- Reminders about task lists (or anything)
- All sorts of warnings about whatever
We've done varying forms of this to differing degrees of success at work.
Dynamic, on-the-fly generation & execution is definitely fascinating to watch in a sandbox, but is far to scary (from a compliance/security/sanity perspective) without spending a lot more time on guardrails.
We do however take note of hallucinated tool calls and have had it suggest an implementation we start with and have several such tools in production now.
It's also useful to spin up any completed agents and interrogate them about what tools they might have found useful during execution (or really any number of other post-process questionnaire you can think of).
>Dynamic, on-the-fly generation & execution is definitely fascinating to watch in a sandbox, but is far to scary (from a compliance/security/sanity perspective) without spending a lot more time on guardrails.
Would love love love to hear more on what you are doing here? This seems super fascinating (and scary). :)
Probably less sensible than you think.
How many terms would they need to do this over? How many terms would they need to do it for _at once_? How many tokens would that add to every prompt that comes in?
Let alone that dynamically modifying the base system prompt would likely break their entire caching mechanism given that caching is based on longest prefix, and I can't imagine that the model's system prompt is somehow excluded from this.
Thank you for saying this out loud. I've been losing my mind wondering where the discussion on this was. LLMs without Tool Use/Function Calling is basically a non starter for anything I want to do.
When I was working with LLMs without function calling I made the scaffold put some information in the system prompt that tells it some JSON-ish syntax it can use to invoke function calls.
It places more of a "mental burden" on the model to output tool calls in your custom format, but it worked enough to be useful.
What do you find difficult about distilling your own prompts?
After any back and forth session I have reasonably good results asking something like "Given this workflow, how could I have prompted this better from the start to get the same results?"
If you understand what rate limiting is, you block them for a period of time. Let's stop being pedantic here.
72 requests per day is nothing and acting like it's mayhem is a bit silly. And for a lot of people would result in them getting possible news slower. Sure OP won't publish that often but their rate limiting is an edge case and should be treated as such. If they're blocked until the next day and nothing gets updated then the only person harmed is OP for being overly bothered by their HTTP logs.
Sure it's their server and they can do whatever they want. But all this does is hurts the people trying to reach their blog.
72 requests per day _per user with a naive feed reader_. This is a small personal blog with no ads that OP is self-hosting on her own hardware, so blocking all this junk traffic is probably saving her money. Plus she's calling attention to how feed readers can be improved!
Even if they had 1000 feed readers which would be a massive amount for a blog, if you can't scale that cheaply, that's on you.
As I pointed out, her blog and rate limiting are an extreme edge case, it would be silly for anyone to put effort into changing their feed reader for a single small blog. It's bad product management.
Of course she can. It's static. She doesn't want and I understand. She's signaling their clients an standard call to say "I think you already have read this, at lest ask me first when this changed the last time".
If every user is collecting 36mb a day like in the story here, your droplet wouldn’t even be capable of serving 500 users a month without hitting your bandwidth limit. With their current rates, your one million requests would cost you around 10 million USD.
That's ridiculously big quantity of data to serve a seldomly updated blog just because the client doesn't want (or know how, or think about) to implement an easy and old http method.
Imagine the petabytes of data transferred through the internet saved if a couple RSS clients added that method.
Yeah, but also... if RSS readers behaved correctly, it would be 512 kb. (170 kb with gzip, if she didn't enable it like you imply – I'm too lazy to check, but I assumed it was on.)
I think making clients behave correctly is much more sustainable solution, although we could do better than doing so at the cost of the end users.
Yews, it's about enforcing their preference on how others should interact with OP's published site feed, on principle. Which is always an uphill battle.
It's about enforcing that people follow standards. Which is still an uphill battle, but at least it's based in something sane. Their work on this has resulted in improvements to a whole slew of popular feed readers that should make life easier for a chunk of the internet, not just OP's own site.
Sounds like you don't know how to scale for cheap.
And since I've ran integrations that connected over 500 companies. I know what a rouge client actually looks like and 72 requests per day and I wouldn't even notice.
> The officials each responded with “that very tired mantra: ‘But the virus is worse,’” Dr. Murphy recalled. “Yes, the virus is worse, but that doesn’t obviate doing research to make sure that there may be other options.”
Not all of them, no. There are going to be side effects or injuries for some people, that’s the unfortunate truth.
But some people are going to think that non-vaccine issues are due to the vaccine due to the proximity of the two events in time. Either issues they already had but hadn’t noticed before getting the vaccine, or total coincidences. Those need to be investigated too rule out possible issues. But even with an obvious explanation people may continue to blame the vaccine.
There are also the group of pseudoscience believers who will truly believe impossible things are the result of the vaccine and may work hard to make their “injury” known.
So I don’t like blindly taking the number of complaints, without even a cursory check for nonsense, and using it in a headline to make it sound like the vaccine’s danger is being covered up, without making it clear how small the percentage is.
Not everyone will file a complaint, true. So even if every complaint was true harm, more people would have been hurt. But billions of doses of the Covid vaccines have been given.
If 20,000 people were injured and “only” one billion doses were given that’s a 1:50,000 ratio. I don’t think the tone of the headline fits a ratio like that.
The CICP has so far paid 11 people a total of about $30,000 for COVID vaccine injuries. One person who suffered myocarditis received a $1,000 payment. If anything, the headline's tone is understated relative to this state of affairs. If only 11 people had ever been seriously harmed by the vaccines, you never would have heard about it.
We had a version of Bomberman that was written in some version of BASIC in high school (maybe QBasic?). It was great because we could hack on it to modify the game and immediately play with those modifications.
Of course it's a stupid amount of money sometimes, but I generally feel like we get what we're paying for.
reply