Ignoring the obvious issue that this whole anonymous story seems suspiciously perfect for selling a related product...
On the one hand...
Companies spent the past couple of decades engaging in various SEO hacks to rank high on search results and OpenAI scraped the internet to train a language model. Theoretically, it seems possible that some of the SEO techniques at least partially colored the flavor of LLM-generated text, and an "AI detector" could pick that up. So if you do a great job writing SEO optimized text (wordy, structured, lots of repeated key words, etc.) you are more likely to be flagged.
But really..
"AI Detector" services are snake oil and will lead to the creation of "Anti AI Detector" services that offer protective spells against the original snake oil. See, we eliminate a bunch of jobs with AI but we create whole new disciplines of work that didn't exist before. "AI Generated Content Obfuscation Specialist - III - W2" coming to a job board near you soon.
I've been thinking along the same lines. The token window IMO should be a conceptual inverted pyramid, where there most recent tokens are retained verbatim but previous iterations are compressed/pooled more and more as the context grows. I'm sure there's some effort/research in this direction. It seems pretty obvious.
Phrase embeddings could bring a 32x reduction in sequence length because:
> Text Embeddings Reveal (Almost) As Much As Text. ... We find that although a naïve model conditioned on the embedding performs poorly, a multi step method that iteratively corrects and re embeds text is able to recover 92% of 32-token text inputs exactly. We train our model to decode text embeddings from two state of the art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.
They are. Moreover, the idea that AI companies are missing and/or not implementing this “obvious” tactic is hilarious. Folks, these approaches have profound consequences for training and inference performance. Y’all aren’t pointing out some low hanging fruit here, lol
Actually, yes I am pointing out low hanging fruit here. These approaches do not have "profound consequences" for inference or training performance. In fact, sentence transformer models run orders of magnitude more quickly. Performance penalties will be small.
Also, I actually have several top NLP conference publications, so I'm not some charlatan when I say these things. I've actually physically used and seen these techniques improve LLM recall. It really actually works.
This...is your blog about prompt engineering. What do you believe this "proves"? How have you blown away current production encoding or attention mechanisms?
Concur. LLM are still very young. We’re barely a year out from the ChatGPT launch. Everyone is iterating like mad. Several stealth companies working on new approaches with the potential to deliver performance leaps.
I think the claim is based on the public political statements made by leaders in Texas. The fact that there is a huge discrepancy in what they say publicly against the science of global warming and the utility of renewables versus what the investment numbers say is the really sad part. Basically it boils down to: I'm going to lie through my teeth to pander to the stupid people who vote for me, but I'm also going to create favorable conditions for my wealthy buddies to make a killing in renewables.
i replied as sibling but deleted it. I think the point to remember is the energy industry in Texas is there to make money. If they can make money with renewables then that's what's going to happen. West Texas has a tremendous amount of wind energy and so... now we have wind farms out there. We also have a lot of flat, sunny, unusable land out in West Texas too so you're starting to see solar farms as well. No one is doing it to SaveThePlanet no matter what the pamphlet says, it's profitable in Texas so it's happening in Texas.
The statements by republicans tend to be a denial of the idea that we should get rid of gas by government fiat, or a skepticism of the ability for renewables to provide consistent power. I've never heard a wholesale dismissal of renewables from most conservatives.
I strongly identify with this position. I don't need the government stopping gas sales. That doesn't mean I don't recognize that renewable energy is cheaper and pollutes less.
What I've noticed on the left is that unless you believe that gasoline and natural gas should be made illegal, you are considered a hypocrite if you also use renewables.
I'm just chasing the rationally superior energy source. The environmental argument doesn't really sway me on this one. And I don't like government coercion for a very useful and life saving substance (fossil fuels).
Nevertheless to directly address your claims. Abbot has been pretty vocal in his support for renewables..his campaign puts out materials extolling his success in bringing them to Texas.
I think what's happened here is that progressives have once again moralized on this issue.
It's not enough to be in favor of and actively take steps to encourage renewables. If you don't do it to the same degree or you don't simultaneously criticize and ban the alternatives, you're presumed guilty of some great transgression.
When people are faced with negative outcomes resulting from things they approve of, they do this passive-aggressive bit where they pretend to have a valid point.
I used to buy into some of this JFK stuff when I was a X-Files watching teenager. What really burst the bubble for me was a documentary I watched where a team of snipers and forensic scientists re-created the exact shot with mannequins with bones and ballistic gel. They didn't even have to try that hard. Using the same rifle and ammo, the first shot they tried resulted in almost the same exact trajectory. I can't find a clip of that exact documentary (circa 2004-2006), but there are others who have done the same. You don't have to look hard to find very comprehensive and scientific explanations for the exact trajectory of that specific shot. But you do have to look very hard to find an actual explanation for why it is impossible that is beyond the level of "golly gee folks, I done shot lots of guns in my life and let me tell you, it ain't possible."
Just in terms of ballistics, the 6.5mm round nose bullet fired by Oswald is not necessarily going to behave like a tapered point 5.56mm or .22LR. Even if I've put a million rounds of .22LR down range into paper targets I'm not magically a forensics expert on a larger bullet's trajectory bouncing around a metal car and through bodies.
I own a 6.5 Carcano and can confirm that the this chambering does exhibit non-standard terminal ballistics when using the old bottle-nosed, flat-base surplus bullets. Also, a lot of these late-19th century surplus rifles (including my 1891 Truppe Speciali) have less than stellar bores, having seen hard use in two world wars. Mine was shot out pretty bad when in service and often keyholes rounds, for example. All kinds of weird things could be possible at impact in that case. That said, it's still a remarkably well-designed cartridge with many features considered modern now, so can be flat-shooting and very accurate. Some detailed analysis of Oswald's Carcano could lend some insight into whether this is relevant.
I would also be interested in a detailed analysis of Oswald's rifle. I don't know how conclusive it would be but it would be interesting. Since it was a surplus rifle there's no telling what sort of wear and tear it had. Same with any ammo recovered at his house or if any was found in the book depository.
Just something like a bottle nose bullet vs tapered is going to affect penetration and ricochets. The range from the book depository to the limo was not really that far, at the point of impact the bullets had significant amounts of energy. They could easily go through bones yet bounce off steel and repenetrate.
I think the Oliver Stone film did too good a job convincing people bullets magically stop when they hit something. Rifle bullets are often very angry and like to make it everyone else's problem.
All I see on that link is them talking about proving that it was possible. When I search, I can't find the actual test, or demonstrating that they can pass a bullet through one block of gel, and then bounce it off 2 "bones" in another block of gel, at 45-degree angles, and cause more wounds, and stay in one piece. If veterans were to comment and say, "Yeah, this kind of thing happens all the time," I might be more sympathetic. But I can't find their actual demonstration, so I don't know.
I do see them quoted as saying "these were not hard shots to make," but no expert riflemen at the FBI could get off 3 accurate shots in the 6.5 seconds it would take to make the Commission's report true.
I've seen "experts" try to tell me that shooting a melon makes it recoil in the direction of the shooter, to attempt disprove the fatal headshot from the front. Again, this flies directly in the face of experience with anyone who has shot guns for fun. This just does not happen. So was it faked? Was it a one time thing? Who knows! It was an "expert," but it sure as heck doesn't square with my experience. And it sure as hell doesn't explain Jackie picking up pieces of John's skull from the trunk lid.
But this really gets to the heart of why we can't agree on anything any more: you can always find an expert who tells you what you think should be true.
If you want to believe that the magic bullet caused several wounds in 2 people, bouncing off bones at sharp angles, then exit the second victim -- it wasn't recovered in Connelly -- and wind up on the gurney of the FIRST victim, in almost pristine form, without even being covered in blood, then I can't help you. It doesn't require credentialed expertise in firearms or ballistics to know that's horse puckey.
Myself, I think Occam's Razor applies here, but not in the way you do. I find it far MORE believable that there was a conspiracy, with multiple shooters, than I do the AMAZING number of ballistical miracles it would take to make the lone gunman story work.
Thank you for replying. I didn't realize it was bolt action. I'm somewhat familiar with bolt action rifles and I understand some of the tradeoffs they make compared to other platforms. Speed is indeed one of them.
I dabble in music production and know some of the people in the "Lofi" world, so I know for a fact that this is not true. It's just a formulaic sub-genre where people are trying to make similar instrumentals with the same vibe. It would be jarring to listen to a playlist while studying and each song had wildly different tempos, instruments, etc.
Also, the music doesn't sound "Lofi" because it's generated by algorithms. A lot of hard work and software goes into taking a clean, pitch-perfect digital signal and making it sound like something playing on a record player from the 70s.
First of all, I'd like to say that this looks like a great project and I wish you the best of luck. I've done a bit of work on building knowledge graphs from semi-structured data and I know that every aspect of it is challenging. Obviously there's the data pipelines, ETL, semantic matching/categorization, statistical models, etc. Just building a simple UI for presenting a large knowledge graph was more challenging than most front end work I've ever done.
Question: if the goal is to build a knowledge graph that can "explain how anything in the world is related to everything else" how do you measure progress toward that goal? And how do you measure the quality? Just having a bunch of topics and relationships is not a great metric in my opinion. Obviously this is still very early, but here's an example I found in about 30 seconds of clicking around:
Thank you so much. We'd love for you to join our Slack community (link on system.com).
Great question. There is no ground truth that we are modeling System after, i.e. there is no causal model of the world out there (to use Pearl's framing). So I'm not sure we can know how far along we are epistemologically. More practically, for the next few years we have plenty of work to just represent all the existing corpuses of scholarship! The truer and arguably more meaningful test of progress though is how decisions are improved — for users, for organizations — that use System.
Re completeness, as I wrote below, System Search results are not necessarily comprehensive — but they will be. System is in the early stages of its development as a public resource and you should expect that knowledge will be missing. The knowledge base will be constantly growing and improving and evolving as knowledge does. Our community will play an important role in relating what we expect or know should be related.
First, thanks! If you'd like to reach out and learn more or talk about your learnings from building something like this we'd be very interested (we have a Slack community and a direct contact form on the site).
As for your questions - we have tools for assessing the reproducibility (in the statistical sense) of models and relationships added to System, as well as tools for users (and built in to the platform itself) to assess the relative statistical strength between any two relationships that you find on the site.
And, yes, we're early on in the process of writing (peer-reviewed) evidence on various topics, and as you note, the value of seeing these systems will grow with how detailed the topics are covered and the overall number of the world's topics shown to be related. I hope you'll stay engaged to see!
Yea those are the expected devices. I think that pretty mimics their botique line which are adorable little synth replicas of the original. They're so cute I had to collect them and have them displayed on shelves in my living room.
Damn, I really wish they would have given these away as VSTs instead. I'd love to bring an official Roland 303 software synth into my setup without paying their ridiculous subscription fees.
That's probably why they aren't free VSTs, this is just for tinkering and nostalgic normies. All the artists making music for public distribution knows its faster and easier to do it all in their DAW
But! FLStudio has Transistor Bass. I don't know if it's available as a VST but IL has posted comparison videos online and I can't really hear any difference through my studio monitors.
DAW are indeed technically superior but the biggest problem is that they thwart creativity. They’re not good for exploring but they’re excellent for executing once you know what you want to do.
> Remember, Stephenson’s target audience consisted of “scientists, mathematicians, engineers, and entrepreneurs.” Given his choice to court private wealth, it’s no surprise that Project Hieroglyph was doomed from the start. After all, you can’t very well expect to succeed as a hero if you stop to ask the villains for their permission.
It's really, really hard to take somebody serious when their political frame of reference makes them see the world in such crisp black and white contrast they just assume, without any further explanation needed, that clearly everybody already agrees that entrepreneurs (or maybe private wealth? As in, non-government wealth?) are the real villains.
Yeah, I feel like there’s this absurd amount of commentary floating on the Internet that basically boils down to “the authors aren’t concerned with what really matters, which is to bring the socialist utopia to life”.
Any kind of societal, economic, human and environmental ill is caused by capitalism, so anything that tries to solve actual problems instead of eliminating or weakening capitalism isn’t doing enough
> Any kind of societal, economic, human and environmental ill is caused by capitalism, so anything that tries to solve actual problems instead of eliminating or weakening capitalism isn’t doing enough
My entire life (40 years) has been feeding me the exact opposite propaganda. In the world I've lived in, capitalism is the answer to all of our ills. Things are only broken (like healthcare or education in the US) because they aren't capitalist enough. It's not really a surprise when the winners of capitalism own the news.
I'm personally glad to see another angle. I thought that was one of the things we liked about the internet.
I'm at 50 years and I remember seeing live television pictures of East Berlin and wondering if what I was watching was black and white or color (black and white TV's were still pretty common) and it was always a shock when there was an unexpected splash of color in an otherwise gray and dystopian hellscape on the TV. Turns out the cameras and everything were color, there just wasn't much color from the environment for them to relay.
For those so quick to shit on capitalism how about proposing something better? Yup, it's flawed, but that doesn't mean it's still not better than anything else we have tried.
And the current flavor of the moment, socialism/communism, sure as hell isn't the answer. Want a good synopsis of how well such systems work in the real world? Grab a copy of the book of "The Hunt for Red October". Skip to the middle where they describe why Ramius's wife died and in particular why it was such a senseless death. Clancy does an awesome job of summarizing the biggest problem with any system that doesn't franchise every participant or provide them with natural incentives to execute their own enlightened self interest.
Yup, there are aspects to capitalism that suck - but the answer isn't to chuck it wholesale for some pie in the sky replacement, but instead work on the aspects that suck. Many of them are well understood - most people can't be bothered to get off their ass and actually do something about it; preferring to bitch online and wait for someone else to fix it for them.
If you want to talk about extreme capitalism (US healthcare) vs. GDR Stazi communism, only, then sure the conclusion that you are lead to is obvious.
And yet there better models than both, once you get past the emotional appeal to this false dichotomy. Any well-off European country with universal healthcare (usually socialised), well-regulated free markets in most other sectors, and relatively low income inequality will do.
That's leaving aside the models that are currently "science fiction".
First of all, healthcare in many European countries is often insurance based.
Yes, in some cases it’s socialized. And in those places the healthcare system is rapidly falling apart/getting worse, and everyone who can afford it goes private.
Just look at the boom in private doctors/insurance in countries like the UK and Denmark.
And of course you also ignore the price paid in the form of taxes, VAT, etc. (Nope, not corporate taxes, which in Europe tend to be lower than in the US.)
If you wanted to nationalize healthcare in the US, you’re basically talking about adding trillions to an already bloated Federal budget.
For some reason, I’ve yet to see a solution to that problem that wasn’t some kind of immature, unproductive slogan throwing like “tax the rich!”
The only reason why the UK NHS is "rapidly falling apart" is the ideology of the Conservative party that's been in power for a decade now. They don't want to have it, but can't say that outright as it would be electoral suicide. So it's the good old Chomskyan Privatisation technique: "defund, make sure things don’t work, people get angry, you hand it over to private capital." i.e. "everyone who can afford it goes private" is not the market triumph that you might think.
Those people who have gone private, _still_ maintain a NHS GP with access to affordable drugs, etc. It's quite unlike private US healthcare.
> If you wanted to nationalize healthcare in the US, you’re basically talking about adding trillions to an already bloated Federal budget.
Nope.
Where did I say that I wanted to do exactly that to the US? Funny thing to exclude "European insurance-based" systems explicitly now, when explicitly including them earlier.
I observed that there are better models, which there is empirical evidence for (1), nothing beyond that.
Any change to USA healthcare would be about _moving_ money not _adding_, would impact an "already bloated" and grossly poor value-for-money private extractive healthcare system: The USA currently spends more and gets less (1). So "adding" is deceptive, and "bloated" is a emotional-noise word in this context.
Yeah, the US has a problem with spending more on healthcare than any other country, but it isn’t a “profit” or “administrative overhead” problem.
(Insurance companies have legally limited profit margins on health insurance.)
Does multiple private insurances add a little administrative overhead? Sure, but nowhere near as much as you see in the OECD figures.
There are many reasons why US healthcare is so expensive.
Some of them have to do with litigation: Skyhigh insurance premiums and a medical “Cover your ass”-culture with unneeded tests and MRIs done on a regular basis.
(In Canada, there is a legal ceiling on how much in damages you can sue a doctor/hospital for. This would be a good idea to implement in the US.)
Another added expense is free riders: All the people that show up in the ER, whether because of the flu or a gunshot, get treated/operated on, and never pay their bill.
Another issue is that the US has some of the best doctors in the world, and new, prohibitively expensive treatments are often introduced here first.
Many, very expensive treatments simply aren’t done (or done very rarely) in countries where the public (fully or in part) pay for treatments.
Waiting lists are another way of limiting costs in European countries. (Waiting lists that can be over a year long, but their healthcare systems are arguably cheaper to run.)
I could bring up some new medicines that in the US cost thousands of dollars per week (Crohns medication for example.) In European countries those medications simply aren’t available, or are only available on an exemption: Meaning that your doctor need to apply to have the cost covered, and is usually turned down.
Another example would be something like SRS surgeries.
These can be extremely expensive, and are covered by insurance, which of course increases the costs of coverage for everyone.
In European countries you’d either face a very long waiting list to get the operation covered, or you wouldn’t be eligible for it. Or only eligible after jumping through extensive hoops.
As someone who lived both in USA and am from EU (can only speak about Slovenian and German system) and I have preexisting condition (asthma and ankylosing spondylitis )
Sure you can find problems in any healthcare system. Because there is no upper limit on spending. There is always something more that you can do.
If you are proper rich* (dozens of millions or more) USA has the best health care system.
Fore everyone else, EU health care is better.
For instance right now I was at my pulmonologist and he wants me to take CT of my chest.
As it is not emergency I could wait for a week (and it will be free), or I can have it today in private clinic, where I copaid 40 EUR*. If it were serious I would have it done for free today.
I got yearly prescription for
Berudual, fosters, spiriva and Singular for free.
Bottom line is yes, sometime there is waiting time, but outside few pathological cases they are quite reasonable.
In USA I had more than double of my current pay, but with rent and my preexisting condition the way that they are, I am actually left with slightly more each month.
And there is another big difference between our system and US. Everyone is insured (weather it's public or private). So there is no fear of ever going to the doctors or calling ambulances.
Private clinics are getting paid by my insurance (I could get free CT at the same clinic a week later), 40EUR fee is just convenience (I don't want to wait) fee. And normally It wouldn't take a week, but with covid and flu season in full swing it takes a bit.
> or I can have it today in private clinic, where I copaid 40 EUR
it seems to me that this is how public healthcare keeps private healthcare honest, by existing.
i.e. you have, and I have also, paid a double-digit sum for the privilege of being seen _today_. But that is not the same as paying for the privilege of being seen _at all_.
The BATNA is very different for the patient. A private doctor can get away with charging a fee for the convenience only, but since it's not the only game in town, they cant gouge to extremes, and charge a huge fee when you need to be seen by a doctor at all costs.
> And of course you also ignore the price paid in the form of taxes, VAT, etc. (Nope, not corporate taxes, which in Europe tend to be lower than in the US.)
The US government pays more for healthcare than the UK government does, and the UK gets universal coverage.
This is fundamentally dishonest of US campaigners against universal coverage: they imply it's going to increase costs and decrease quality, when all the evidence is that it would decrease cost and increase both length and quality of life.
Funny how the people always banging on about the evils of capitalism have never actually lived under a communist system.
And also seem to believe there’s some sort of socialist Utopia in Europe/Scandinavia, despite the fact that social democracy has been dead there for decades.
Then again, in general anti capitalists are a fairly ignorant bunch, who’ve never let reality get in the way of their dogmatic politics.
On the one hand... Companies spent the past couple of decades engaging in various SEO hacks to rank high on search results and OpenAI scraped the internet to train a language model. Theoretically, it seems possible that some of the SEO techniques at least partially colored the flavor of LLM-generated text, and an "AI detector" could pick that up. So if you do a great job writing SEO optimized text (wordy, structured, lots of repeated key words, etc.) you are more likely to be flagged.
But really.. "AI Detector" services are snake oil and will lead to the creation of "Anti AI Detector" services that offer protective spells against the original snake oil. See, we eliminate a bunch of jobs with AI but we create whole new disciplines of work that didn't exist before. "AI Generated Content Obfuscation Specialist - III - W2" coming to a job board near you soon.