Technically AWS isn't doing anything wrong, but the optics are bad. I am genuinely concerned about the future of established Open Source projects.
Amazon Web Services has responded to Elastic adopting more-restrictive software licenses by simply forking the latter's Elasticsearch and Kibana products with an open-source license. This basically means developers have a choice: use software developed by Elastic that has a somewhat limited license, or an open-source offshoot …
I think one could equally worry about the future of new Open Source projects - why bother if you're simply giving a leg up to your massive potential competitors.
But it also works the other way - there wouldn't be the manpower to work on some crucial Open Source projects if large corporations weren't paying their staff to do it.
Open Source only really works if there's enough mutual interest between a disparate group of developers. It's hard for anyone to have a mutual interest with a corporation the size of Amazon, but licensing is probably not the answer: they have the resources to take a useful concept and reimplement it if it's worth their while. And if you insist on finding an answer, you might find yourself approaching software patent territory,,,
> I think one could equally worry about the future of new Open Source projects - why bother if you're simply giving a leg up to your massive potential competitors.
For me, I think this view is misunderstanding the key principles of Open Source.
My take on Open Source (in this context) is that it's a deliberate pushback against the way patents and/or copyright have been used to monopolise access to goods and services, which in turn has both driven up costs for the consumer and stifled innovation.
I.e. Open Source isn't intended to make you rich. Instead, it's meant to enrich society by making sure that both knowledge and capabilities are freely distributed.
In fact, to quote Wikipedia, that wholly accurate (and open source) fount of all knowledge:
Being organized as effectively a "consumers' cooperative", open source eliminates some of the access costs of consumers and creators of derivative works by reducing the restrictions of copyright. Basic economic theory predicts that lower costs would lead to higher consumption and also more frequent creation of derivative works.
I've emphasised the key point there.
So if someone falls back into the mindset of "I can't make this open source, as my competitors will then be able to use it", they're falling back into the protectionist/selfish mindset which is what gave rise to open source in the first place.
And it's also missing the other point somewhat, in that any competitors are subject to the same limitations. I.e. any improvements they make to open source software also has to be released back into the wild, so that other people can make use for it.
It's also worth noting that socio-economic growth has often been at it's greatest when countries have decided to ignore copyright and patent laws. E.g. the USA during the industrial revolution and China over the last few decades.
Sadly, once enough people have become rich through this approach, they tend to adopt the same protectionist leanings as the people they "stole" from...
Admittedly, the Open Source model perhaps starts to break down a bit when you're dealing with an established monopoly, since they'll be able to bring their existing resources and/or economies of scale to bear - and I've even heard it said that monopolies can sometimes *gasp* abuse their dominant position. Perish the thought!.
Equally, while a monopoly is still just as beholden to contributing back to any open source it uses, it's likely that any contributions they make will be weighted towards their business processes and ethos. And this in turn could end up making the open source less useful for others.
And on the slightly mutated third hand, I'm in no way saying that all patents and copyrights should be abolished!
There is definitely something to be said for extending at least some protection to creators, so that they have an opportunity to profit from their creativity - or at the very least, to have some control over how their creations are used.
But at the same time, I do think we've travelled far from the intent of the original copyright and patent laws. Certainly, I suspect "life plus 70 years" is far from what the original writers of the US copyright clause meant when they declared the need to "securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries"...
User "juice" writes:
My take on Open Source (in this context) is that it's a deliberate pushback against the way patents and/or copyright have been used to monopolise access to goods and services, which in turn has both driven up costs for the consumer and stifled innovation.Almost perfectly correct, except rhat the goals and ideals you ascribe to Open Source are actually those of its predecessor, the Free Software movement. The younger sibling Open Source movement, in turn, is the result of trying to unite these goals and ideals with "intended to make you rich”.
I.e. Open Source isn't intended to make you rich. Instead, it's meant to enrich society by making sure that both knowledge and capabilities are freely distributed.
My own provisional take on the whole saga -- the 20-yr-plus saga of Free Software / Open Source, not this current spat -- is that perhaps spats like this shows they can't be united. (Or, OTOH, with Elastic Search valuated at ~15 billion dollars, perhaps they can... But then the ES founders would by most reasonable definitions already have achieved the goal of becoming rich, so WTF are they still fighting about? Not everyone can be Jeff Bezos. Hmm, naah, I'll stick with my original take.)
You are only highlighting two paths. One that presumes that the "project" is really a company, pursuing the usual corporate interests of dominating markets and making money. The other that SOME open source projects depend on the generosity of revenue generating entities to support developers so they don't have to send their commits from behind a dumpster at Starbucks. While the second is a fair point, we didn't get here based on VC startups with a bait and switch "free" and "open" version that pivots to restricted or fully closed licensing, or corporations starting to step up for supporting large scale projects that had been domain of universities before them.
There's way more happening in the community. In this case both Amazon and Elastic are both right in parts and wrong in general. Elastic is basically saying that Amazon has lots of money, and they want it(in the form of coding hours and other things). Amazon isn't paying for SUDO or ssh, or a zillion other pieces of open code, so elastics position is a shakedown from their perspective. Amazon for it's part has been acting in ways that squash a smaller company and its attempt to compete in enterprise hosting, which they should be more careful of these days, but Elastic chose to build its product as and from open source code with a permissive license.
But the reality is, by creating their project as an open source project, they don't own it. The community of users and developers do. So Elastic can wallow in denial about facing the music for choosing a flawed business model, but they have no right to complain. If the community as a whole or any given user exercises their right to fork to maintain their own use of the code, they gave them that right.
Elastics argument against amazon is inside out, as it presumes their company alone is allowed to claim the top slot for offering hosted services based on the code they don't themselves own. This happened because a lot of the projects users are based in AWS, so it made sense for them to run it the same cloud.
The real victims here are the rest of the community, who now have to ride out the churn as the developers and contributors pick sides. Worse, the road Elastic is going down is a evolutionary dead end. Even if it "wins" the fork war, where does it go next? Rubbing sand in the eyes of every user working off a cloud platform isn't a business model that works long term, even if they get an Antitrust settlement out of the deal. Unfortunately the "corporate fork" may also shed key developers and lose the vital spark that drives innovation forward. That's not much of an issue for a mature, feature complete solution, but search-at-scale still feels like a moving target with room to grow.
Not "technically": AWS genuinely has done the right thing here.
AWS runs (and sells) Elasticsearch as a service. It also contributes its improvements back upstream. Via OpenDistro it also contributed further functionality, some of which was only otherwise available as paid-for add-ons (e.g. alerting).
Elastic.co didn't want AWS's software contributions. It wanted their money.
AWS won't be held over a barrel by Elastic.co. Open source software without license fees is what allows AWS to scale up and up and up. This applies to their whole stack, from the Linux kernel upwards. At their scale, it's cheaper for them to employ their own software engineers rather than pay licences, in the same way that it's cheaper to build their own servers than buy from Dell or HP.
In itself this makes no difference to Elastic.co, since they were never going to see any money from AWS anyway. What they now risk is that all the other unpaid community developers will jump ship to the AWS fork - and that their future potential customers may pick this one too. Elastic will have to make its value proposition the support services which come from buying a commercial partner - which they could have done all along.
IMO, the open source community has *gained* a new product champion for Elasticsearch in AWS. We all know AWS's business model doesn't include selling software licences, so there's no risk that AWS will pull the same trick as Elastic.co. There is a potential risk that they will steer it in the direction of their own interests, but since their own interests include deploying Elasticsearch at massive scale with high reliability, that's likely to coincide with the community's interests too.
Aside: Elastic.co no doubt noticed that AWS sells services with built-in licence fees for Windows and SQL Server and VMware and Oracle, and hoped AWS would do the same for them. That ain't going to happen. AWS resell those particular pieces of software, not because they can't write their own database or virtualization layer (they have!!), but because a subset of customers insist on running those exact pieces of software. And crucially, those pieces of software have been closed-source and commercial from day one, so there's nothing to fork.
EDIT: it's also worth noting that in keeping Elasticsearch development fully open source, AWS is helping *its own competitors* - both the other big cloud providers, and the more specialised Elasticsearch service providers. AWS don't mind. They continue to compete on reliability, breadth, and price.
I think a lot will come down to what happens next. If AWS work hard with a respected open source umbrella organisation to put in a proper governance structure and build a community in which they are a major but not the only contributor, it will look a lot better.
Elasticsearch has never had this community, really, which is why the tricks Elastic are pulling are possible. Don’t get me wrong, it’s an incredible product and building it was dependent on a lot of VC funding which would not have been possible to raise if they didn’t have control.
But no one really wants to be in a position where they come to depend purely on Amazon for the fork, particularly if they want to deploy on other clouds or on prem, distribute as part of a product or run as part of a SaaS setup.
While there is no doubt a risk of major open source projects being held hostage by a few major players, tinkering with the licences is unlikely to change anything. The open source principles have proved their worth time and again and many of the large companies have embraced them so much that they provide stewardship and resources for some projects. Companies that release their software under an open source licence should not be surprised if their competitors adopt it: it was their choice to do so. There is a risk that they can use such projects to dominate the market. If this is the case the regulation may be required, just as it would in any other case of market dominance.
there is windows, oracle db, vmware and other proprietary SW in the cloud.
I think business plans depending on an exclusive right to open source in perpetuity is the problem.
They opened source to get credibility from customers as a vendor, but then there always was the threat that a better vendor comes along.
I'm not sure AWS size matters in this specific case.
> Now they are black holes which will swallow any new valid open source project.
Really? How many new open-source projects are only feasible because they can be ran on a cloud architecture? I suspect there's probably been at least half a dozen new "AI" business ventures spun up on AWS in the time it's taken me to write this sentence.
Certainly, it'll be hard for any newcomers to compete with existing cloud providers, simply because of the scales involved and the fact that there's a lot of expensive hardware (and support thereof) underpinning their services. But that's not the fault of open source software, nor is it something that open source software can solve (or be expected to solve).
But at the same time, most of these clouds are running on open source software. So the tools for someone else to try and do the same are available. Certainly, it's a lot more feasible than (say) my mate Bob deciding to buy his own freighter so that he can compete with Maersk for international shipping and distribution...
Banon ought to check his history; this sort of manoeuvre rarely works out well. Open Source is too big a movement now to be worried by a sudden (why are they all sudden or unexpected changes?) licence change like this.
Redhat just changed is licence conditions after canning CentOS but it could have announced both changes at the same time. They didn't which makes the latest announcement look like a knee jerk reaction.
Similarly, QT has just thrown a spanner in the works. The Qt Company is moving Qt 5.15 to "Commercial-Only" LTS Phase starting on Jan 5th 2021.I think that this will mark the of the end of a very beneficial cooperative community relationship which could become more confrontational.
AWS don't give a shit about anything apart from money. Will they open source their own tech? no - only some bits that no-one gives a crap about. If their whole eco system was O/S then I would buy it. They play the O/S card but that is just a bunch of crap cause they know if divides the community. Their goal is to destroy any competition using any means necessary. Run everyone else's shit cheaper than they can themselves and pick up the consumer cash. Its totally unfair and but the options are limited - change licenses... go commercial.... go and get a job for amazon? Mega monopoly and bullying at global scale..... if this was a bank then things would be different.
On the other hand, Amazon are continuing to provide their fork under the AL 2.0, so they are benefiting the community by keeping their fork available. As long as they continue to do this into the future, there is no harm to the community, and just the repositories change.
The only way that Elastic. are going to survive this is if they put some killer, can't-live-without features in their next release of ElasticSearch.
On your last point about harming the competition, US corporate law almost mandates that a US publicly listed company attempt to out-compete their rivals. although the line between intense competition and anti-competitive trading is a fine one that often needs the courts to decide.
"Amazon are continuing to provide their fork under the AL 2.0, so they are benefiting the community by keeping their fork available..."
Gosh yes so kind of them. Nobody else could possibly fork the project and make it available publicly. If only there were some sort of service where you could go to provide a fork of a Git project, perhaps a centralized location or "hub" or perhaps a place where you can continue to merge Git patches like in a science "lab". Gosh I'm so sorry I'm rambling crazy talk now. Such a place would have to provide continuous integration servers, issue tracking, all kinds of stuff, and it'd have to be free. Thank god for Amazon!
Forking a repo to a personal account on , perhaps a centralized location or "hub" or perhaps a place where you can continue to merge Git patches like in a science "lab".
I'm not seeing the issue, AWS forks the ES code, as AWS make money integrating hosted ES with their IAAS - that's a corporate sponsor with a github repo - https://github.com/aws
Just boycott Amazon. We're building a large SCADA system in the cloud. None of it is on Amazon. Because they are shady bastards who cannot be trusted. Not only do they not play fair wrt Open Source, but they will simply collude with others to shut your business down if they don't like you, as we have just seen recently.
Who can trust a business that operates like that? What happens if our venture becomes very successful and very profitable? We'd expect a call from Amazon execs saying "Nice little earner you've built there. Running on our kit we see. Shame if something happened to it."
While many will applaud you for your ethics at not using Amazon because they are 'shady'(no worse than the others IMO) , you'd probably want to be careful at most companies with that mindset.
I could deploy my systems into Azure or GCP.
But I don't.... why? Because the employer wants the best-bang-per-buck platform for the job (at this time).
Not many will give a toss about the OSS movement and the impact Amazon has on it..
If I was to take your view, and enforce it at my employer - I'd be out the door quicker than you can say AWS.
That would be a constant over most employers... (unless you yourself own the company)
> but they will simply collude with others to shut your business down if they don't like you, as we have just seen recently
Now I *know* you can't be referring to Parler given that the court filings are out for the world to read - even Parler didn't put any real effort into claiming there was collusion.
So, care to link to the example? Would be an interesting read
> What happens if our venture becomes very successful and very profitable? We'd expect a call from Amazon execs saying "Nice little earner you've built there. Running on our kit we see. Shame if something happened to it."
That's why multi-cloud is a thing. There's little functional difference between AWS suddenly becoming the mafia, and them having a service wide outage. Which is much, much more likely that the scenario you've painted.
I *do* hope, though, that you're not building it on GCP - Google having previously shown they *are* willing to something vaguely similar to what you're claiming.
much as i hate to defend the dread pirate bezos.
Running the SW at scale and fixing the issues that occur is what AWS kick back.
The product AWS offers works with terraform, uses the same auth system, and has a published api supported by the cli. Just installing ES on a linux box doesn't do that, so why do they care that you are installing from the same sources.
They 100% don't care about the community, but it's smart business for them to push the code back upstream.
> This basically means developers have a choice: use software developed by Elastic that has a somewhat limited license, or an open-source offshoot developed by a gigantic technology company that also offers it as the Amazon Elasticsearch Service in the cloud.
So the choice is: be screwed now, or be screwed later.
Instead, new versions of the software will be offered under the Elastic License (which limits how it can be used) or the Server Side Public License (which has requirements that make it unacceptable to many in the open source community).
Especially the takers. :)
In order to ensure open source versions of both packages remain available and well supported, including in our own offerings, we are announcing today that AWS will step up to create and maintain a ALv2-licensed fork of open source Elasticsearch and Kibana.
Wow, how nice of you AWS!! I hope you gave them a big cuddle first before making it bigger.
Hmmm, what's that smouldering over there? You should probably go put that out.....
Bows before the new Microsoft; The circle has fully turned
To any other startups listening. CHOOSE YOUR LICENCES CAREFULLY.
"CHOOSE YOUR LICENCES CAREFULLY"
More or less what I was going to say. They should have understood the implications of the licence they used. No doubt they saw OSS as a trendy way to get into the market and maybe pick up some contributions and fixes but it had long term consequences that they should have anticipated.
Sorry guys, I get that it must be frustrating to see some huge cash-rich monstrosity of a corporation grow even richer off the backs of your efforts and not contribute one penny of cash or one line of code back to your project, but that's exactly what the license you released the code under says they can do. That's why it's called a permissive license.
Now instead of having them be a parasite you now have them as a direct competitor. Attempting to shove the genie back on the bottle was never going to work, this was always going to be the outcome.
Should Amazon do the right thing and pay back into a project they're benefiting so much from? Yes, probably. That would be good karma. Must they? Absolutely not.
Remember, when you start on an open source project, the license you start releasing code under will in most cases be the license you're stuck with for the lifetime of the project. A change in license will inevitably lead to a fork. That's not a bug, it's a feature.
Absolutely this. Think very carefully about your choice of license.
Don't just follow the herd if you're going to care about how and when others can use and distribute your software. Equally, do *not* come up with your own special-snowflake license unless you have *very* good reasons...
"Should Amazon do the right thing and pay back into a project they're benefiting so much from? Yes"
And have they? Yes. It hasn't been by paying Elastic a bunch of cash, which I'm sure Elastic would have preferred, but AWS upstreams a bunch of code to the Elastic project. Mostly because they're lazy, just like most of us; if you have to maintain your own version or release your changes as the canonical version, the latter is easier. Still, AWS fixed some things and gave us the code, which we now get to use for free if we want to. If I start to use Elastic, I'll profit from Amazon's changes and I can guarantee you I won't be paying Amazon for that. That's one of the primary benefits of the open source and free software philosophies: the advances made by one user get made available to everyone else without someone chasing the users for extra payments or license audits.
With a stock-market valuation of ~15 bn $, I'd say Elastic is also pretty much a "huge cash-rich monstrosity of a corporation". They're trying to paint themselves as "the little guy" being unfairly trampled by a behemoth, but they're just a somewhat smaller behemoth.
Oh, and also:
Remember, when you start on an open source project, the license you start releasing code under will in most cases be the license you're stuck with for the lifetime of the project. A change in license will inevitably lead to a fork. That's not a bug, it's a feature.Unless you start from the other end, with the most restrictive license you would consider using. Later going from less to more open shouldn't scare anyone off back to your older closed-source version, AFAICS.
Can't have your cake and eat it
I'm a bit pissed off at Elastic if I'm honest. I'm a contributor to their OSS product (albeit not massively, just some bits here and there) but even so, I'm unable to use certain features of their product unless I pay them $$$ ... so what, you want it to be open source? Or you don't?
If I knew then what I know now, I may not have been so enthused to contribute to the project.
I was actually delighted when I saw Amazon take the product and release their own version as it finally meant I'd be able to use other features in the product, knowing the changes are supported by a big enough company that they're not going to "just disappear" as happens with other forks.
If I was a contributor (I'm not btw), and a company was making money off my contributions (by their licensing), the sensible thing would be for all companies who are using my code (however small a contribution) to offer me a free perpetual licence for their full product.
However, we know this will not happen...
So, if Elastic can change their licence on their product, why could I not change my licence to them on my contribution?
> So, if Elastic can change their licence on their product, why could I not change my licence to them on my contribution?
Quite a while back, a project that I (and others) had contributed to under GPL wanted to relicense the codebase as LGPL, so needed buy in from contributors.
Most were against it. It's not that the reasons given for wanting to change the license were necessarily crap, so much as most didn't feel the outweighed the objections they had to using LGPL as a license (using LGPL would have allowed it to be used in closed source products - as long as it was linked dynamically).
Many of the objectors had been willing to contribute because the believed in the protections the GPL gives.
But, despite the objections, the change was forced through (and the development agreement updated to make it easier to relicense in future).
The result? Contributions dried up and the mailing list became eerily quiet.
There are sometimes good reasons for wanting to change license, but if you're making it less free* then it's almost always going to signal bad things for the future of your project.
*speech, not beer
I guess it is very difficult for a company to struggle for revenue and see others profiting from their work. And the license change rarely does any good to the company. Definitely not helping here.
I tend to be very very very cautious about open source software developed by a for profit company, as the temptations are way too much. I would rather use open source software under some sort of organisation. Python and Postgresql come to mind. I am sure there are way more. Though libreoffice has started doing weird things with version 7. Oh well.
Java can be tricky specially if idiots in the company (no other way to say it) start using the non free versions or non free options bundled with free versions. And you are kept unaware. I pretty much had to check all the machines. Now, I flat refuse to use anything with JVM, that keeps my brain relatively sane. Has worked till now. Had to dump Cassandra and move to (happy days are here again!) Postgresql.