I had LLM (Claude) work with OTF to generate an entire infrastructure HCL (from existing). It built a very nice project that seemed idiomatic from my experience.
Then used it over the course of several hours to refactor it to take variables/inputs for everything, then over a few days got it to a state where it would create entire new environments "equivalent" to the original environment. Days because you know... it's TF in AWS which is slow, so the round-trips were probably 90% of the wall-clock time here.
I'm not a hardcore veteran Infra eng, but I'm decent, and I was able to do way more with LLMs than if I'd had to do it myself.
I am with you on this. I am a good, and experienced infra engineer. And I feel like - no know that - llm can probably replace me if a good operator handles that.
I don't know about theatres, but I do know about hotel rooms.
If you lower the price too much, you get a different sort of clientele. The sort of person who wrecks the place and annoys all the other patrons nearby.
Then the cleanup costs a lot. Often more than the amount of revenue collected on the room.
It absolutely makes more sense to keep the hotel room empty than to lower the price to keep it fully occupied.
First, I am also frustrated by companies trying to prevent unauthorised used.
But second, the reasons are:
(1) For AI company, someone publishing: "I asked the model a question about crime, and it talked shit about black people! Look! [damning quote that you can also get model to say/do]." Stability took the "let people do what they will" tack and now Forbes and every other major media mouthpiece slams them at every opportunity about how they are ethically-challenged.
(2) For Replika, someone chatting with their online girlfriend: "I love you more than my wife and children." Then someone hacking Replika exposing these conversations, and now Replika is in hot water because all these divorces. Replace example with 100 other similarly awful situations like talking about mental health problems, crimes, petty squabbles with their coworkers, or political problems.
Forbes can write that crap, but the problem is with people who make decisions basing on that. I wonder who are these people that care about all this nonsense.
Start searching SuperHOT and RoPE together. 8k-32k context length on regular old Llama models that were originally intended to only have 2k context lengths.
Any trick which is not doing full quadratic attention cripples a models ability to reason "in the middle" more than they already are crippled. Good long context length models are currently a mirage. This is why no one is seriously using GPT-4-32k or Claude-100k in production right now.
Edit: even if it's doing full attention like the commentator says, turns out that's not good enough! https://arxiv.org/abs/2307.03172
"You ship it, you take the pager. Once it's stable, then SRE org will take the feature. If it gets unstable again, SRE will hand it back."
If someone vibe codes something, and it works, then no reason not to merge it. So just set it up so if it doesn't work, they're on the line to fix it.
Along with their oh-so-supportive manager.
But also, if you have the clout, doing what you're doing nips the problem in the bud earlier, and so is more efficient. Good that you have the clout.
reply