It used to be that distributed systems were a big trade off. They were operationally complex, they had limited apis (NoSQL), but they scaled. The best solution used to be to build things using a non-scalable but easy to use and run system, and then re-write it later if it needed to scale (often in a big hurry).
This is just not the case any more, though. Why? Two reasons:
1. We’ve gotten much better at distributed systems so the apis aren’t nearly as limited. It’s no longer that you either choose hard-to-use things like Hadoop/NoSQL or elegant but unscalable single-server databases. You can have both good abstractions and scale.
2. The cloud makes it possible to get systems as a service so there should be way less ops than running a single node system yourself
In the case of Kafka, I’m super biased as I’m one of the original authors, but I think the abstractions Kafka gives, stream processing capabilities, connectors, etc are just way better than a lot of the traditional solutions. Using something worse until you “need” Kafka might make sense on premise, but not in the cloud.
Confluent offers a Kafka service which is fully managed so you don’t do any of the upgrades, security patches, midnight pages, etc you just use the APIs. This is super affordable for the kind of simple apps the article describes. The price varies by cloud, but e.g. on GCP it starts at $0.11/GB for reads and $0.10/GB stored. That is a lot cheaper than using a single node system and then rebuilding everything if you need to scale, but not only that, it is also lower operational overhead (effective none) and a better interface/abstraction.
I think this isn’t unique to Kafka, either. There are great managed systems that are built to scale for most of the kinds of data systems you would use—-CockroachDB, Spanner, Aurora, Snowflake, Elasticsearch, Bigquery, etc.
Basically, you can have nice things now, just like the big tech companies.
The problem I see with kafka is that it was built before cloud architectures were commonly adopted (with distributed systems everywhere). Confluent has put a lot of effort dragging kafka's architecture to the present, but some major features are missing:
- Auto-scaling: Confluent finally introduced "elastic scaling" a few months ago but it only allows you to scale up and must be triggered by the admin (no threshold-based auto-scaling).
- Multi-tenancy: Planning for a multi-tenant kafka cluster is not for the faint of heart. Achieving isolation tends toward liberal usage of topics of which starts to become unmanageable in the low thousands. This isn't crazy when you've got a few hundred microservices and several tenants to keep isolated.
- Decoupled brokers and storage: Any broker scaling or failure can lead to downtime while event storage is redistributed.
Confluent's Cloud service reduces operational overhead but isn't always feasible due to cost, resource limits (like service accounts or schemas for instance), data controls, etc.
At my last startup, things would have been far less dysfunctional if the leadership team had read (and adopted) this philosophy. When we got backed by Venture funding it basically forced us into rapid scaling that we weren't prepared for and most of the engineering team quit because of mismanagement.
The point about effective managers need to stay out of the details and don't solve their team's problems can't be overstated.
This is Jay, the CEO of Confluent. No, actually quite the opposite. We took pains to ensure that the license was written so that software products like Landoop were free to embed our open source and compete with us in that way. This FAQ covers the competition in more detail: https://www.confluent.io/confluent-community-license-faq
I am no expert on this, but based on Bryan's analysis, if Landoop were to release a SAAS offering of their product, they would be in breach of the license. As Landoop are pushing Kafka on Kubernetes, I assume that is their strategy - to have a SAAS offering at some stage (we are all going to the cloud, i thought?).
Do you care to support that assertion? It reads very much like one.
> BY INSTALLING, DOWNLOADING, ACCESSING, USING OR DISTRIBUTING ANY OF THE SOFTWARE, YOU AGREE TO THE TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE TO SUCH TERMS AND CONDITIONS, YOU MUST NOT USE THE SOFTWARE. IF YOU ARE RECEIVING THE SOFTWARE ON BEHALF OF A LEGAL ENTITY, YOU REPRESENT AND WARRANT THAT YOU HAVE THE ACTUAL AUTHORITY TO AGREE TO THE TERMS AND CONDITIONS OF THIS AGREEMENT ON BEHALF OF SUCH ENTITY. “Licensee” means you, an individual, or the entity on whose behalf you are receiving the Software.
"... you agree to the terms and conditions of this agreement" seems like a dead giveaway for an EULA -- specifically, an attempt to bind the licensee by contract law in addition to copyright law.
There's confusion of terms here. You seem to be using "EULA" to mean "contract". If you look at the FAQ Confluent added, they read "EULA" to mean something more specific.
"End User License Agreement" has no reliable, specific meaning in the industry. "EULA" for short gets thrown away even more willy-nilly, in all kinds of circumstances. I've seen it used for SaaS terms.
You're certainly right that "EULA" is ambiguous -- but Bryan provided a more specific definition which was pretty clearly concerned with the "contract law in addition to copyright law" part:
> EULAs are an attempt to get out of copyright law — where the copyright owner is quite limited in the rights afforded to them as to how the content is consumed — and into contract law, where there are many fewer such limits. And EULAs have accordingly historically restricted (or tried to restrict) all sorts of uses like benchmarking, reverse engineering, running with competitive products (or, say, being used by a competitor to make competitive products), and so on.
For example, that's why a railway operator (a private company) can fine you when you cannot present a valid ticket. When you get on the train, this action creates a transportation contract between you and the railway operator.
Contract law is a large and complex area of legal code, but the key term that get repeated is the "meeting of minds" which is the core disagreement between those that find EULA valid and those that find them invalid. Classical contract law holds that you can not make contract binding if one party has not read it or do not understand if for one reason or an other. This is used as the example when someone talk about switching a contract under the table, using microscopic hidden text, or other contract schemes. By using excessive length, language, complexity, and a position of power (you may not use the property you bought unless you agree to this additional arbitrarily terms) many see it as identical to switching the contract under table.
There is also some additional fun extras in contract law, like the concept of fair terms. This very old idea is that a contract should be a balanced deal. Terms that unfairly give one party undue benefits can then be challenged as invalid.
> There is also some additional fun extras in contract law, like the concept of fair terms. This very old idea is that a contract should be a balanced deal. Terms that unfairly give one party undue benefits can then be challenged as invalid.
US and state laws have a concept of "unconscionability" for extreme cases, as well as prohibitions on penalties, as opposed to prior agreements to fix damages based on reasonable estimates, and so on. But "the deal wasn't fair" isn't any general defense against breach of contract claims. Courts generally avoid digging into the business or other advisability of contracts. They err on the side of giving parties the deals they agreed to.
Perhaps you were speaking from the perspective of a different jurisdiction?
In some, but very few circumstances. In particular, I don't think the railway operator example works in the UK, at least not automatically, because, otherwise, why would they have special legislation relating to "penalty fares"? Look for the legislation on http://www.legislation.gov.uk/ if you're interested.
Here's an illustrative example that may or may not accurately correspond to the current state of the law in any particular jurisdiction, but it show why this implied contract stuff doesn't work in practice.
A farmer might put up a big notice saying: "By camping overnight on this field you agree to pay me £100 per vehicle per night." Then a bunch of travellers might camp there. Can the farmer sue the travellers for the £100 per vehicle per night? My guess is that he could only sue them for damaging his grass, because, despite what the notice said, the travellers didn't "agree"; they camped illegally and without permission, like they usually do. Probably they smashed up the notice to show what they thought of it. Therefore the farmer can only sue them for trespassing, not for breach of contract. (He might also sue them for the cost of replacing the notice if they did smash it up!)
Likewise, you could put a notice on the front door and every window of your house saying that by breaking into the house burglars agree to pay you X pounds. Would it help you in any way? Of course it wouldn't. You'd get laughed out of court if you tried it.
I don't think you should give this the short shrift you are - in particular, right to use vs other rights is not the defining characteristic of EULA's. There are EULA's for redistributable components and have been for a very long time (for example, the MSVC runtime libraries).
The most defining characteristic of a EULA is usually the "licensed but not sold" part.
I suspect adding this particular entry does more harm than good for you - it makes you look like you ignoring the meaningful argument here.
Personally, I'd remove it until you have some reasonable response.
EULA has no specific meaning in industry. It's been used for so many different kinds of terms, in so many different contexts, that it's lost nearly all meaning. Like "OEM".
> The most defining characteristic of a EULA is usually the "licensed but not sold" part.
Public licenses for software, say MIT or BSD, also arguably license, rather than sell. But they still include disclaimers of warranties, like merchantability and fitness for particular purpose, implied by the Uniform Commercial Code, which governs contracts. Huh?
At least under the US law I've seen, from Jacobsen to Hancom, there's no hard, meaningful legal distinction between contract and license, as some activists theorized early on. Even when those activists drafted licenses, like the GPLs, that explicitly claimed to be licenses and not contracts, they still included contract-like disclaimers and limits on liability.
Realistically, license and contract rules coexist and overlap. How do we interpret license terms? By rules of contract construction. What claims do plaintiffs make for violations? Copyright infringement and breach of contract. What makes a license irrevocable without consideration? Promissory estoppel, a contract doctrine.
But do you view the Confluent Community License as a contract? And do I own the copy (not the copyright!) of the software that I git clone'd, modified and built?
Jay, your FAQ says “EULA only gives you the right to use; the Confluent Community License grants other rights as well.”
This is factually untrue. Many EULAs grant additional rights, such as the right to make a backup copy of the software. In the case of Freeware (ah, halcyon days!) the EULA often allowed redistribution of copies.
I can address some of the critiques that seemed aimed at the blog post I wrote announcing the Confluent license change:
1. We aren't trying to get cloud providers to license our proprietary features. We run a cloud service of our software.
2. The book analogy is not very accurate. We have an FAQ here the helps clarify interpretation. The limitations it places are extraordinarily small, 99.9999% of users are completely unimpacted, it really only impacts companies wanting to offer, say, KSQL-as-a-service. https://www.confluent.io/confluent-community-license-faq
3. We aren't trying to "co-opt" the community or open terminology. We actually tried super hard both in the license and in the blog post to be honest and upfront. Whatever else you think you have to agree that Confluent's license is _exceptionally_ permissive and the software has a pretty great community of users. How do you describe a license that let's you run, modify, fork, and redistribute the code and do virtually anything other than offer a competing SaaS offering of the product?
4. Bryan Cantrill is an amazing engineer, but, well, as a lawyer, I think ours are probably better. We're quite confident in the enforceability, but it's a bit ironic because I remember this being the FUD around GPL that it was "totally unenforceable".
5. The "open source companies are all failing"-meme isn't factually correct. Many open source companies are actually doing quite well. MongoDB has gone up in value about 3x over the last year, Elastic was the breakout IPO of the year. There are a handful of other really strong businesses a year or so behind, including Confluent. An open source project is not in-and-of-itself a business model, but it is, just empirically, a big part of some of the recent successes in the infrastructure space. Probably worth noting that the reverse is true too: if you look at some of the really cool up-and-coming open source platform data technologies, a lot of them have the support of a company behind them. Of course there are plenty of sucky open source companies, but that is true of every category of startup.
6. I agree that it is silly to moralize about the behavior of the cloud providers. They are following their economic interest. The point is that this behavior does undermine the cycle of investment in some of the more promising hard tech open source projects and to try to change this dynamic.
7. This article has a bit of a tone of "Son, new things aren't possible, trust me, I tried them and have the scars to prove it". I have huge respect for Bryan, and I know that to some extent that is his schtick as a public personality, but I'm not sure that attitude is most likely to lead to improvement. I don't think the current crop of licenses was handed down from the mountain on Stone Tablets by our elders to be revered and not questioned. I think CockroachDB, Elastic, MongoDB, and Confluent are building really innovating technology platforms and building pretty cool companies to help fund that. I don't think we need dogma. And I still don't say "GNU/Linux".
> How do you describe a license that let's you run, modify, fork, and redistribute the code and do virtually anything other than offer a competing SaaS offering of the product?
A proprietary software license. Let's not forget the infamous "don't be evil" clause.
> The "open source companies are all failing"-meme isn't factually correct.
Several of the companies you have mentioned (including yourselves) are no longer "open source companies" since you now develop proprietary software. You might not consider this a failure (maybe a "pivot"), but you are no longer an "open source company".
Don't get me wrong, I completely believe that there is a financial problem caused by cloud providers not paying you for your development work. And I understand the frustration and lack of fairness in such a dynamic. But that doesn't change that you now develop proprietary software.
> I don't think the current crop of licenses was handed down from the mountain on Stone Tablets by our elders to be revered and not questioned.
Nobody is claiming that, and those licenses have changed over the years. But the changes have always come from the community. MPLv2 was written so that it could be integrated with GPL code. The GPLv3 was written to deal with concerns about locked-down hardware. The AGPLv3 was based on a community fork of GPLv2.
The new proprietary licenses are coming from companies that wish to protect their businesses. This is clearly a different dynamic, and I think it's quite unfair to paint your critics with the brush of being unquestioningly reverent of our elders -- when in fact we are seeing that the existing, gradual evolution of licenses by the community has been co-opted by companies wishing to protect their own interests.
It would be more accurate to describe it as proprietary than it would be to describe it as free software or open source. Proprietary software is software which restricts your freedoms when it comes to the usage, modification, or distribution of said software. If you prefer, you can also use the term source-available to distinguish the degree of restrictions -- but the point is the same. There are restrictions on your freedom in the software and thus it is proprietary.
Not everything has a middle ground. Software is either proprietary (restricts your freedom) or it isn't -- and discussions about how proprietary it is (how many restrictions it imposes on users) are secondary.
I could argue a completely different case. The only "restriction" it is placing on you is that you may not restrict anyone else from exercising the same rights that you yourself were granted by the license, which I believe is the original spirit of the 4 freedoms and the GPL family of licenses.
The software is "effectively free," because for every user who simply uses it for personal use, research, or even many forms of commercial use, they have all of the same abilities that they would have with any other free software license.
The restriction only comes in when you make a derived work of the software and do not pay forward that derived work under equivalent licensing terms as the work on which it was based.
And this is where the real disagreement is. What exactly is a "derived work", and where do you draw the line in the sand?
If I'm essentially selling access to somebody else's software, I have little doubt that access software constitutes as a derived work. I think it's fair that a license like the SSPL asks me to release the code which provides access to the free software as free software itself.
Suggesting that "My freedoms are being restricted" because a licensing term prevents you from restricting the freedom of others is the same argument that "permissive" license proponents argue against strong copyleft licenses.
If I release something as SSPL, it isn't because I'm trying to "restrict your freedoms". It's that I'm trying to prevent you from restricting other's freedoms by selling them proprietary work based on it.
The license being discussed here is not the SSPL. It's the Confluent Community License, which does not have any of the GPL-like aspects you refer to. Instead it simply denies the use of the software (freedom #0) for an "Excluded Purpose" (creating a competing product to Confluent). I'm sure you'll agree this is not in any way in the original spirit of the four freedoms.
> What exactly is a "derived work", and where do you draw the line in the sand?
This is mostly determined by copyright law, since "derived work" is a legal term of art.
> If I release something as SSPL, it isn't because I'm trying to "restrict your freedoms". It's that I'm trying to prevent you from restricting other's freedoms by selling them proprietary work based on it.
This is the justification, but due to the design of the license it is de-facto impossible to actually comply with its requirements. Therefore it acts as a de-facto proprietary license. Many copyleft lawyers have stated that the license would likely require you to re-license Linux under the SSPL if you run SSPL code on a Linux server. This is not possible to do, and thus you are forced to pay MongoDB to get a business license.
Maybe there is a place for a license like the SSPL, but given how there would be effectively no company that could comply with it (even if it didn't require relicensing to SSPL, many companies have contracted code that they cannot relicense to a free software license) I fear it would have the same effect.
> Many copyleft lawyers have stated that the license would likely require you to re-license Linux under the SSPL if you run SSPL code on a Linux server.
There's no such thing as a "copyleft lawyer". Even if there were, there wouldn't be many of us, even if you counted every one, worldwide.
I personally don't agree with the reading you referred to. But if Mongo's SSPLv2, which they've submitted to OSI, is any indication, it won't be tenable much longer.
A licence is proprietary if it restricts any if your four freedoms. The Confluent licence restricts freedom 0, the freedom to use it for any purpose. It is therefore proprietary. JSON's license (with the "The Software shall be used for Good, not Evil." clause) is proprietary too.
Now, we can have a discussion over the degree of proprietary-ness, but I disagree with the statement that it isn't proprietary. Of course it is different to some other proprietary licenses, but I believe that discussion is secondary to the discussion over whether it is proprietary.
You can say the license isn't "open source." The term has a well-defined meaning provided by the OSI, and they arguably have the right to define what it means and which licenses meet the definition, being the ones who pretty much invented the term.
You DO NOT, however, get to also define the meaning of the word "proprietary." The English language is not your plaything, and you have not been given dictatorial rights to re-define words as you wish. "Proprietary" does not suddenly mean "restricts any of the four freedoms" just because you said so. When antt calls this instance a misuse of the word, [s]he is relying on the common English meaning of the term, which very much supports their point. Your rebuttal is pretty much "nuh-uh because we're now using a different definition."
> The term has a well-defined meaning provided by the OSI, and they arguably have the right to define what it means and which licenses meet the definition, being the ones who pretty much invented the term.
Funnily enough, many people would argue the exact opposite -- that "open source" has a common meaning that is separate from the "Open Source" which the OSI defines. I don't really have a strong opinion either way.
> "Proprietary" does not suddenly mean "restricts any of the four freedoms" just because you said so.
I am using the term in the same manner as the FSF. Maybe you disagree with their definition, but it's hardly something I've just come up with in this argument -- this definition in the context of software licenses has been in use since the 80s. If you disagree with that definition, complain to the FSF about their subversion of language instead of me.
> If you disagree with that definition, complain to the FSF about their subversion of language instead of me.
Whether you follow the FSF's lead is up to you!
The FSF itself changed the way it talks about these issues. Its "philosophical" writing used to distinguish "semi-free" or "source-available" and "proprietary". They even had a nice diagram showing semi-free as a middle ground.
At some point, they made a rhetorical decision to lump everything east of "free software" together in one "proprietary" pile. I wish I knew why. But the reason couldn't have been precision.
> You DO NOT, however, get to also define the meaning of the word "proprietary."
And to clarify that I am not defining the meaning of proprietary, I'm using the definition the FSF uses (and has used for significantly longer than the span of this comment thread). Whether you think that's a reasonable definition is a different point, but I was being accused of (effectively) moving the goal-posts.
On the other hand, you _are_, possibly intentionally, mixing the concept of open-source as defined by the OSI, and proprietary software as defined by the FSF. You're acting as if proprietary software is the antonym of OSS, and citing the FSF definition to support it. Except, the FSF doesn't talk about open-source (except to say that it "misses the point"), it talks about Free Software. And the OSI doesn't provide a definition of proprietary software.
You can't run your code on a users computer without respecting the other three freedoms, and the AGPL goes even further and says you can't even run it on your own computers.
This was an argument I used to hear being made loudly and unironically by the MIT/BSD crowd in the 90s/00s.
To quote Stalman, free software isn't about having the source available any more than a library is about making books with movable type. It is about giving people the power to be programmers without selling their souls to Big Evil.
That essentially all the code that makes Amazon Amazon is DevOps code on how production code and hardware is managed is something no one could have seen in 2007.
Pretending that orchestration is not the most important part of the stack today is as ignorant as saying that source code doesn't matter because you have the binary version was in 1995, again something I've heard said unironically.
The AGPL, the most radical of the free software licenses, does not deal with the supporting code on how to deploy the software. The prosperity license does, because it's written by people who are in the trenches today. And it's completely free when you open source your full stack.
> You can't run your code on a users computer without respecting the other three freedoms, and the AGPL goes even further and says you can't even run it on your own computers
Absolutely wrong for the gpl, which is a distribution license, not a usage license. This is a well established fact.
I've cancelled the mongodb standardization in the big company I work for specifically to avoid them.
The only sane way for a cautious company to use those vaguely licensed software is the proprietary one. That's the intent of this artificial grey area.
And as a consumer I try to avoid proprietary software whenever possible. License management is a huge pain.
Middle ground exists because it is ultimately for a court to decide whether or not something is a copyright violation. These things aren't black and white. A license is a piece of legal advice which suggests the decision a court might make if it were to be taken to one, based on past judicial decisions. When it comes to new technologies and new distribution methods (or distribution loopholes), there's not much precedent to go off. The uncertainty here is the middle ground.
You are focused on enforcement which (might be interesting in an internet lawyering way) is irrelevant if noone uses the codebases in the first place because of the license makes it unuseful for users and coders.
Almost all of these "middle ground" license cannot be combined with the normal licenses that have huge functioning communities.
even transistors, the basic underpinning of our digital worlds, have middle ground - it's the norm, not the exception, and dichotomies are almost entirely a human fiction to make things more computationally tractable
So we can go down rabbit holes like this I guess? I mean whats the middle ground between believing the earth is flat and the center of the universe vs a modern scientific view?
Metaphors aren't really a good thinking tool here.
Instead maybe asking (as I suggested you do) if EVEN ONE of these LICENSE schemes is actually working for the community its supposed to serve vs the pr blahblah we see in their announcements?
Confluent continues to fund the development of quite a lot of Apache 2.0 code and that is a huge part of our business and strategy. Perhaps your point is that if a company produces any non-open source code they are not an open source company?
Microsoft and Oracle also fund the development of a lot of open source code, but I think it'd be a bit of a stretch to call them open source companies.
I’ll never understand why we collectively wag our fingers at individuals or companies that try to keep the likes of Amazon from building a profitable service off of their hard work then contribute back little-to-nothing. Would redis, mongodb and dgraph have even considered alternate licensing if companies like Amazon had thrown them a minuscule amount of funding and patches? We can’t know because they didn’t. And then we sneer at them for having the audacity to try to stay open but stop these giants from using them and throwing them away.
This kind of aggressive behavior toward these people trying to stop their own destruction at the hands of the biggest and most profitable companies in the world makes me wonder what the hell is wrong with our industry. This could easily be any of us. It might be any of us in the future. What do we gain by consolidating control in 3-4 different giants? What are we achieving with such a black and white view of what constitutes open source or not?
This is nothing new. The loudest people in the software industry are the ones with the most time, which by definition is the ones who are least employed or most activist.
Back in the real world that reddis wants to charge a license fee would just mean an email to your boss and accounting with a short message saying "This is a better alternative since we can change the source code as needed for a pittance". Then it will go to the CEO and he will sign off with "Done" after reading the first 20 words of the report.
I agree that the proliferation of new licenses is annoying. But I think the reason you see changes in licensing from Elastic, MongoDB, etc is that the world has changed pretty dramatically since most of the original licenses were created. This proliferation existed early on in the days of open source and eventually died down to a set of standard things that solved the problems most people had. But the reality is the public cloud has dramatically changed the dynamic and many of the people who want to make code available with a permissive license don't have an option that is common and well understood and that protects them. I believe such a thing will likely emerge and will allow us all to standardize.
> many of the people who want to make code available with a permissive license don't have an option that is common and well understood and that protects them.
The AGPL is that option. It is a copyleft license that explicitly regulates the 'public performance' of software (putting conditions on public performance of a copyrighted work is a right of all copyright owners, there's nothing "exotic" involved here) in order to close the perceived 'service provider loophole' where software that's made available as part of an application services platform is not required to make its complete source code available, including even modifications that are purportedly asserted to be "for internal use".
Short answer:
1. AGPL doesn't actually solve this problem. This is why MongoDB just moved away from AGPL.
2. AGPL is quite restrictive for people who want to use the software in proprietary applications but don't want to open source their own code. This is a really large proportion of usage for us and we don't want to restrict people in that way.
Say that your SaaS product to share pictures of cats uses database X, which is AGPL licensed. Depending on how you read the license, your SaaS product could have to be licensed as AGPL. Or not.
That ambiguity is why many companies won't touch the AGPL with a mile long pole.
No, this has no effect on Kafka, which is part of the ASF and under the Apache 2.0 license. This just impacts things like KSQL that are part of Confluent Platform.
"We’re changing the license for some of the open source components of Confluent Platform from Apache 2.0 to the Confluent Community License."
This implies that the components are still open source. However, as you probably know from the open source definition: https://opensource.org/osd-annotated your new license does not qualify as "open source" by the most commonly understood definition because it violates article 6 "No Discrimination Against Fields of Endeavor." Specifically, you're not allowing your software to be used in a capacity similar to how you use it yourself.
Wouldn't it be less disingenuous to write "We’re changing the license for some of the previously open source components of Confluent Platform from Apache 2.0 to our own partially proprietary Confluent Community License." Or maybe instead of partially proprietary, shared source, or choose your own word for not really open source?
Edit: My original comment was based on the original blog post, which has now changed. Props to the author for being responsive to feedback.
Could you be more explicit and say that it means the software isn't "open source"? Complying with OSI's definition of "open source" is what "open source" means. They defined the term, so they have a prerogative on what it means.
"Visible source" might be a good description if you want to call it something, but don't confuse your users with "open" and "community" and other feel-good terms.
Why not just use Affero GNU General Public License v3 for these? It seems like a lot of you guys are using ASL 2.0 (a permissive license) and getting upset when people fork and proprietarize the code for their services and solutions.
Aside from MongoDB, who seems to be changing the license simply to kill off MongoDB's vendor ecosystem to benefit itself, everyone else seems to be using ASL 2.0 and then regretting the natural consequences of doing so.
The usage of the AGPLv3 license seems to be enough to ward off large SaaS competitors (Amazon and Google stay away from areas that are heavily AGPLv3 or similar).
This is a great question. Many people think the AGPL solves this problem but it absolutely does not. This is why MongoDB, which was AGPL licensed, just changed to a custom license.
The other reason we don't want to use AGPL is that it can be quite aggressive in making you open source your own code if you want to embed our code in a proprietary application. We actually think building proprietary applications is a fine use of our software, and is one of the major use cases for this kind of infrastructure. This is addressed in more depth along with many other questions in this FAQ:
https://www.confluent.io/confluent-community-license-faq
> This is a great question. Many people think the AGPL solves this problem but it absolutely does not. This is why MongoDB, which was AGPL licensed, just changed to a custom license.
If the goal is to kill off all potential vendors participating in your ecosystem, then yes, AGPL doesn't work. But if the goal is to prevent cloud provider abuse (e.g. Google, AWS, Azure, etc.), it works well enough.
> The other reason we don't want to use AGPL is that it can be quite aggressive in making you open source your own code if you want to embed our code in a proprietary application.
If you wanted to avoid that, you could grant an exception to be slightly more permissive for that purpose or to otherwise clarify the grant of use.
I know I personally would appreciate it if vendors who don't like the consequences of using ASL 2.0 would consider this option as an alternative.
And of course, selling full exceptions to AGPL in its entirety is a valid business model option.
Hey Jay. Just want to say good job on the execution on this. It's clearly articulated and goes along way to show the industry how and why this needs to happen.
I also give it a +1. There's a lot of debate presently around what I call the "Little Red Hen" problem. Everyone wants to "eat the bread" when y'all are done, but very few people want to help grind the grain or knead the dough.
There's a document gathering people's thoughts on the current state of Open Source here, started by Jesse Anderson (formerly Cloudera):
Yes the idempotence part of this feature set is very similar to TCP (the transactional consumption and updates obviously aren't). But this isn't a reimplementation at all. TCP provides deduplication within the context of a connection tied to a process. If that connection is lost or the process dies then duplicates may occur. The feature in Kafka is much stronger as the "connection" is persistent and replicated with the log so effectively the "connection" fails over if the server dies.
That is part of it but there is also a general purpose transaction feature that lets you link together updates, state journalling and consumption all in a single transaction. This enables correct stream processing on top of Kafka and is arguably the more technically sophisticated aspect.
This comment in tech crunch article is misleading and you may want to clarify on that. I agree that this is a nice feature for kafka streams or applications having consume-transform-produce loop. I as user/dev like kafka because of the design choices taken to keep things simple and much effective for users.
“It’s kind of insane. It’s been an open problem for so many years and Kafka has solved it — but how do we know it actually works?” she asked echoing the doubts of the community.
It used to be that distributed systems were a big trade off. They were operationally complex, they had limited apis (NoSQL), but they scaled. The best solution used to be to build things using a non-scalable but easy to use and run system, and then re-write it later if it needed to scale (often in a big hurry).
This is just not the case any more, though. Why? Two reasons:
1. We’ve gotten much better at distributed systems so the apis aren’t nearly as limited. It’s no longer that you either choose hard-to-use things like Hadoop/NoSQL or elegant but unscalable single-server databases. You can have both good abstractions and scale.
2. The cloud makes it possible to get systems as a service so there should be way less ops than running a single node system yourself
In the case of Kafka, I’m super biased as I’m one of the original authors, but I think the abstractions Kafka gives, stream processing capabilities, connectors, etc are just way better than a lot of the traditional solutions. Using something worse until you “need” Kafka might make sense on premise, but not in the cloud.
Confluent offers a Kafka service which is fully managed so you don’t do any of the upgrades, security patches, midnight pages, etc you just use the APIs. This is super affordable for the kind of simple apps the article describes. The price varies by cloud, but e.g. on GCP it starts at $0.11/GB for reads and $0.10/GB stored. That is a lot cheaper than using a single node system and then rebuilding everything if you need to scale, but not only that, it is also lower operational overhead (effective none) and a better interface/abstraction.
I think this isn’t unique to Kafka, either. There are great managed systems that are built to scale for most of the kinds of data systems you would use—-CockroachDB, Spanner, Aurora, Snowflake, Elasticsearch, Bigquery, etc.
Basically, you can have nice things now, just like the big tech companies.