Does the model need to offer new testable hypothesis if it provides a way of explaining existing results that current models can't?
If it is competing against another model that does both that and offers new testable hypothesis (which experiments match), the other model is the clear winner. But lacking that, if no other model explains all existing data, is new testability really necessary when it is the only model that currently explains all existing tests?
That said, aren't most of theoretical models only contenders for such, as in they haven't been expanded to actually explain all testing results, only that, as far as they have been expanded, there are no contradictions yet? So they need physicists to expand them, but if the model is wrong, the effort might largely be wasted, and we have some models that there is disdain for not because they contradict existing experiments, but because they have eaten too many careers without showing value in return?
Isn't this a double plus good phrase? What makes this more responsible? Reasoning about first order effects of different disclosure models? But what if someone uses higher order reasoning and critical thinking to reach a conclusion that other disclosure models are better for the average user and the long term health of the industry, even if they are worse in any individual case. A difference in the security culture incentivized by different disclosure patterns. Why does this one win the name of responsible while other alternatives, which have never been proven to be worse, are automatically marked as irresponsible?
Reminds me a bit of the concept of identity theft, as a way to say that even though the bank (or other creditor) was the one who had money taken from them, it is actually the random person not involved in the transaction who is the victim and has to hold the debt until the issue is resolved.
It's a security industry term. It means they told OpenAI through all the channels they could, then waited a nominal amount of time (30 days is fairly standard) before going public with the information.
The other side would be irresponsible disclosure. Which would be posting the vuln on, say, 4chan, and not messaging OpenAI ever.
Could you elaborate on what other disclosure models you're referring to? I can't imagine something being "more responsible" for the public than privately notifying the owning party to give them time to fix the issue, before notifying the rest of the world (including malicious actors) about it.
Didn't the original authors end up leaking this before OpenAI fixed it? They gave them a chance, but then had to decide between staying fully silent or publishing the details despite malicious actors learning about it before it was fixed or leaving users in the dark. They chose it was better to warn users and inform malicious actors despite it not being fixed.
>This vulnerability was responsibly disclosed to OpenAI. Despite multiple follow-ups, we received no communication beyond an automated reply to our initial disclosure. OpenAI's documentation fails to describe sensitive capabilities granted to the model (e.g., running privileged scripts) or risks of model manipulation via indirect prompt injection, instead focusing solely on functional limitations and data-handling concerns. As such, we are publishing our findings to enable informed decision-making regarding the risk surface.
That very last sentence was considered justification of putting this knowledge into the wild when OpenAI refused to fix it. So, if we consider it justified with a delay, then we are saying it is acceptable (it is "responsible") to give the information to malicious actors as long as you tried to warn the right party first.
Compare that to two alternatives. Alternative 1 is never disclosing it to the public until fixed. Saying it is never acceptable to let malicious actors know until it is no longer a concern, even though this will mean users are kept in the dark about the risk.
Alternative 2 is to reduce that timeline to 0. Say that users are immediately warned, despite the risks of making it known to bad actors.
So if we are saying the current delay is acceptable, but both a longer and a shorter delay are unacceptable, then why is that? What justifies the current delay, what makes that the responsible one, rather than a shorter or longer window?
>I can't imagine something being "more responsible" for the public than privately notifying the owning party to give them time to fix the issue, before notifying the rest of the world (including malicious actors) about it.
What about ensure they have fixed it, and only considering it responsible to disclose it when fixed (alternative 1)? If it is never fixed, then the bug is never disclosed, because it is not acceptable to tell malicious actors how to exploit a vulnerability? Even evidence of use wouldn't be justified, as publishing this makes all malicious actors aware of it rather than just a subset of them.
And if you disagree and think some window is reasonable, then apply that argument to a slightly shorter window and repeat until either the argument hits some built in limit or reaches a window of 0.
Nothing of the level of rocket failure, but I've tracked down issues where you are never sure of the cause. You keep the doubt and let it drive you. You aren't as much sure of a theory, as you have the theory you most want to disprove and keep failing to do so. The more you fail to disprove a given theory while other people with their own personal 'targets' do end up disproving them, the more you can report that the theory is the reasonable conclusion. But you never given up the idea of looking to disprove it. Eventually others join you and work to disprove your theory. As the group continues to fail to disprove it, it becomes the officially stated cause unless someone can provide evidence otherwise.
Sometimes I'll have one that I'm stuck on for a month before finally disproving it, and it is an interesting feeling. There is some level of happiness I succeeded at my goal, but it is very bittersweet because it normally was my last working theory and now I'm simply lost until I can formulate a new one. Sometimes disappointment in myself that I might've missed some easy way to disprove it for so long, but other times the way to disprove it was sufficiently hard enough that I just accept it is what it is.
Consider indie games. If there are 10 of them and 5 are great, you don't need any filter. You look through 5 great and 5 not so great games and end up with 5 great ones.
Now go to a world where indie games explode. But only 1 in every 100 are great. There are now 100,000 games, but most qualify as very low quality. There are now 1,000 great games (and a few of these might be the perfect game you dreamt of), but if you don't have a filter and are buried under 10s of thousands of horrible games, things feel worse.
With a filter, you now live in a world where you can easily find most of those great games with only a few lower quality ones showing up. So as long as the filters that exist, whatever they might be, can handle it, more is better even if quality drops.
Unless the quality extremely fast, say my previous example of 100,000 games but only 1 in a million was a great game. I think this level of quality drop is extremely unlikely. Instead, I suspect the real problem is if the filters can keep up, because they depend upon human effort, so it is possible to hit a point where they are overwhelmed and stop functioning properly. That's when things get worse. As long as the filters hold, more building leads to better outcomes even with a drop in quality.
Isn't this still assuming we can even determine what is true or false?
Newtonian physics is false, but it works well enough we teach it in college. But our best models of physics are currently in disagreement, so can we even say they are true? Given the replication crisis, especially in social sciences, how many of peer reviewed findings can be called true? Even experimental results can be false (consider studies that found FTL neutrinos, which were rejected as an error in the experiment, and which was eventually confirmed but it took quite a lot of work and in a softer field than physics with a claim less absurd than FTL, would have likely long been accepted as a true finding).
Even in math, basic statements aren't really true or false, but more a question of "given these axioms, can we prove or disprove it" noting that we have different systems with different axioms. If we are talking basic sets, most people are using naive set theory which is inherently contradictory, which means that notions like true or false probably can't be considered well defined.
Newtonian physics doesn't just work well enough for education. It provides an incredibly accurate and precise model of the world except at extremes. The majority of engineering does not necessitate using theories of relativity. Both theories are incomplete models approximating reality and are very far from being false.
True and False in general communication means based on best available evidence and expertise statement contains no obvious contradictions or falsehoods based on an optimistic parsing of meaning language and intent. Notably this leaves out misleading or missing data because those concerns are separate from truth and falsehood.
E.g. if I say the earth is round we optimistically parse round to include oblate spheroid and rate it true.
If I say that the earth is flat we rate it as false because there is no reasonable interpretation possible other than confusion or malice.
If it was so clearly ineffective, why does it get challenged more often and replaced? Existing corporations aren't likely to change, but new startups and work owned coops exist, so why don't they compete?
Maybe ranking it on a scale of best to worse is too simplistic a view, and there are reasons this develops. Maybe it is the best option when there is a good leader, thus such structures dominate, much as a government ran by philosopher kings are better. But this only lasts as long as a wise rule is in charge, and it reverts back to a norm, and eventually, due to pure time and chance, enough bad leaders come on board that slowly dismantle the giants, but this happens at a time scale we don't particularly notice due to how much inertia large corporations can have (before we even get into the less pleasant issues like regulatory capture).
>If it was so clearly ineffective, why does it get challenged more often and replaced?
I supposed you meant "why doesn't it get challenged"?
Well, look at how long it took for a democratic/Republican system to appear and survive. The French 1st Republic was immediately at war with all of Europe (I am not talking of Napoleon at all here, it was before that, when the French King was executed).
Nowadays, good luck getting any kind of financing with an "alternative" governance model. The banks and investors will either refuse or edge by pushing higher return rates on you. The whole system is conservative.
The adage "democracy is the worst system, apart from all others" only becomes true long-term. There are plenty of short-lived democracies back to antiquity, in the middle of the middle ages, during the Renaissance, the XIXth century... All stamped down by "more efficient" dictatorial empires... That aren't here anymore. You can expect the same in the even more cutthroat corporate environment, where fitting the system buys you leverage.
And don't get me stated on startups: most of them seek only an exist strategy. Very few challenge any existing behemoth. They are basically externalized R&D.
One other question I had but wasn't sure if it would leave my previous post too unfocused is "aren't we a bit too early to determine in our current government systems are really the most effective?" This is something that will be decided by political scientists far removed from the current societies who can see how our current societies evolve.
Think the difference between AI saying "This paragraph seemed muddled and lacks a clear point. Consider rewriting it." vs "Here, I rewrote this paragraph to focus it more on bridging the previous and next paragraphs."
The problem with this as a metric is that it is loosely defined so it becomes quite easy for a person to twist it to justify almost any level of AI usage as "well, it is still more effort than <X>".
>This is just the modern equivalent of "just google it".
I don't think that is always the case. Sometimes it is. Other times the social cues and later follow up makes it seem the person thought they were being really helpful, not sarcastic, by sending the response. Yet other times, the person acts as if it was their own response and not the AI's, almost akin to passing off the AI's work as their own.
This is most notable when the original question shows effort was put into it and it isn't a simple case.
I can't judge the specific situation. But if this happens to you a lot, I'd suggest looking at how you are asking things from other people (i.e. how you communicate).
> original question shows effort was put into it
What matters is how the other side interprets it, not your level of effort or your expectations. If the other side apparently doesn't get what you wanted to happen, that's a communication issue.
Because being right 60% of the time with minimal work is still amazing, as long as one accounts for the failure rate correctly.
Say I want to look up some game from my childhood, which I barely remember any details for. Going to google and trying is likely going to be very difficult unless I happen to get lucky with some key element. But if an LLM can get it right even a minority of the time, it can lead to me quickly finding the game I'm looking for.
This does depend upon the ability to evaluate the answer, like checking against source or some other option where you know a good answer from bad. If you can't, then it does become much more dangerous. Perhaps part of the reason AI seem to empower experts more than novices in some domains?
>Are you also disillusioned with professional sports, music, acting, and art?
Not the person you were asking, but I think we need to double down on disillusionment in these. I've spoken to too many kids who dreamed of careers in this well into high school, often at cost to other academic paths, when their performance already clearly showed they weren't going this route. Sadly, it is hard to be strong about correcting kids because it is seen as not believing in them and not encouraging them.
As disillusioned as one might become in academia, the path one is on to get there tends to better align with setting students up for a successful career outside of it compared to the ones you listed.
Some kids who try to compete in a winner-take-most market, whether that's being a famous artist, performer, or academic, will succeed; most won't. No one who doesn't try will succeed. Someone is going to succeed. (Who wrote The Teacher's Argument? People who by definition made a famous musical, that's who.)
The thing is, the cost of discouraging the wrong kid -- the one who ends up curing cancer or otherwise innovating in an extremely useful area -- is unbounded.
The cost of encouraging the ones who fail can be heavy, but at least it's finite.
And it's not always obvious if "their performance already clearly shows they aren't going this route." The Nobel archives are full of acceptance speeches that describe how the recipient got off to a slow or unpromising start.
> The thing is, the cost of discouraging the wrong kid -- the one who ends up curing cancer or otherwise innovating in an extremely useful area -- is unbounded.
> The cost of encouraging the ones who fail can be heavy, but at least it's finite.
Assuming that every kid has a non-zero chance of being the "right kid," then discouraging only one child results in infinite cost and so every child should be encouraged to try to cure cancer...
If it is competing against another model that does both that and offers new testable hypothesis (which experiments match), the other model is the clear winner. But lacking that, if no other model explains all existing data, is new testability really necessary when it is the only model that currently explains all existing tests?
That said, aren't most of theoretical models only contenders for such, as in they haven't been expanded to actually explain all testing results, only that, as far as they have been expanded, there are no contradictions yet? So they need physicists to expand them, but if the model is wrong, the effort might largely be wasted, and we have some models that there is disdain for not because they contradict existing experiments, but because they have eaten too many careers without showing value in return?
reply