Elastic amends Elasticsearch Python client so it won't work with forks then blocks comments

Elastic has modified the official Python client for its Elasticsearch database not to work with forked versions, and closed the GitHub issue to comments. Elasticsearch is a database manager and analytics engine, often used for log analytics. The elasticsearch-py client has over 202,000 downloads and is described as the " …

[Geez Money]

Ugh...

I deeply dislike what both sides have done here. I've always thought open source was a bit naive not to see these things coming. On the other hand it's still a dick move by a company Amazon's size if they're not doing their part to contrib back. And lastly while the license change makes sense the connect check just seems pointless. It's so easy to patch out, all it's doing is encouraging half maintained forks, using old versions and other security nightmares.

[doublelayer]

Re: Ugh...

How generous Amazon should be is a very subjective issue. They did contribute code fixes back, and they didn't keep any enhancements to themselves (that would have weakened their business because Amazon Elastic wouldn't be completely compatible with others' Elastic installs). I don't know if that's enough, but it isn't nothing and it is what most open source projects already work with. I would have liked it had Amazon also chosen to donate lots of money to the other maintainers. However, in this case, Elastic also benefits from the work of other maintainers by selling their commercial licenses for the software. They do not pay those external maintainers. It seems hypocritical to me to switch the license on those maintainers for optional generosity which they too choose not to do.

[Anonymous Coward]

Re: Ugh...

Also Elasticsearch benefits because AWS supports it.

Plus, the phrase 'blocks comments' is never ever good.

Changing the licenses and hiding from the debate makes people wonder what is next.

[RichardBarrell]

Re: Ugh...

> They did contribute code fixes back, and they didn't keep any enhancements to themselves

FWIW, I've seen with my own eyes redacted stack frames in the error logs when AWS's hosted ES crashed on me. AWS's ES fork is certainly not open and they are very definitely not contributing everything back upstream.

When I say "crashed", I mean in spectacular fashion. AWS's ES is incredibly unreliable in my experience. I would not voluntarily use it in production. When I looked at the support forums, AWS had for more than half a year had a known broken build of the cluster software set as the default to be deployed when creating a new ES cluster. No idea if they even fixed it since. Amazon Elasticsearch Service is radioactive poo and you should not touch it. Run away, do not walk, run.

I'm a tad sympathetic to Elastic here because on their public forums they've been getting plagued with requests for help caused by the bugs in AWS's closed fork.

[heyrick]

Re: Ugh...

"I've always thought open source was a bit naive not to see these things coming."

Indeed. Most of human progress through the ages has been people walking on the backs of others. If you are involved in open source, it should be because you enjoy it, want geek cred, or have an employer that pays you to do it. If you're going to get stressed and start changing the terms of use, it's not really open source any more is it? It's "available but with restrictions", and adding checks into the code itself (more restrictions that as Geez noted, could be patched around, which would be an incentive not to update), we all know how that story usually ends up.

[Anonymous Coward]

Re: Ugh...

"If you're going to get stressed and start changing the terms of use, it's not really open source any more is it?"

You do known what open source is don't you?

It isn't that here is the source code, do with it as you please.

It is, here is the source code, do with it as the license says. If it says you can not use it without buying a license, then so be it. If it says any changes that you make have to be open sourced, so be it. The code is still open source.

It it says it needs a license, then not free (as in beer) software.

[doublelayer]

Re: Ugh...

"It is, here is the source code, do with it as the license says."

The license said Apache 2.0. Then they changed it so they could demand more money. What Amazon did was in compliance with the Apache license.

You will also find that there are restrictions on what a license can say before the FSF and OSI will decide it no longer qualifies as "free" and "open" respectively. You don't have to care about their definitions, but they are generally accepted arbittors of such things. You must pay to use the software is not approved by either because it places restrictions on who may use the software. Just keep that in mind.

[Anonymous Coward]

Re: Ugh...

It said apache 2.0, now it doesn't. Contributions before the change are still apache 2.0, the source is still available.

Is Chef no longer open source?

They changed their terms around 2 years ago, added an EULA on an apache 2.0 license, no commercial use without a paid license.

Is Red Hat not open source?

Requiring paying for software doesn't place a restriction on who can use it. You can chose to pay for it or not. Its not saying, if you are under 165cm you can not use this software. That is restricting who can use it, that can not be overcome. Saying, if you are a business, you need to pay to use it, doesn't make the business unable to use it. They can, if they get a license.

Open source does not mean no need to pay.

[FIA]

Re: Ugh...

You must pay to use the software is not approved by either because it places restrictions on who may use the software. Just keep that in mind.

I thought the GPL was OSI/FSF approved??

It says explicitly that you may charge for the software; section 4, when applied to the covered work:

You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee.

It is perfectly legal for me to sell you a GPL program for a million dollars, until you buy it I am under no obligation to give you the source code for free.

However, once you have purchased that program the GPL then would obligate me to give you the source code for 'free' (other than reasonable costs incurred doing so).

Section 6:

6. Conveying Non-Source Forms.

You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License,[...]

You could then make copies of the software compiled from that source code available for free if you so desired. (assuming you didn't infringle on trademarks, etc. etc.)

(aside: I get it's the Apache licence in use here, just wanted to point out that 'allowed to do something' and 'for no money' aren't the same thing. ;) )

[doublelayer]

Re: Ugh...

You are conflating two different things. I said that "You must pay" is not approved. The GPL says "You may charge". The two are not the same. "You may charge" means that I can refuse to give you the software unless you pay me money for it. However, if you do pay me money for it and get a copy, you can continue to use it without paying me again and you can give it away for free. Those actions are specifically mentioned in the GPL as well. In a "you must pay" situation, if you continued to use it or gave it to someone else without paying me, you would be in violation. The first is supported. The second is not. You can still easily do it and lots of projects do. You just won't get FSF and OSI approval.

[foxyshadis]

Re: Ugh...

The other part of this is that GPL says *You* may charge, but you may not restrict anyone else from distributing it as they wish. So your customers can just go to the one guy who paid you and get it from them, instead, and you can do nothing about it, except making getting it from you so attractive that no one wants to go elsewhere.

Though all of this discussion is about distribution of source and/or binaries, when ElasticSearch's license now embeds charges for certain *uses* of them, which isn't covered by GPL so much as Stallman's opinions over the years.

[Charlie Clark]

Re: Ugh...

If you look back at the history of open source you'll generally see that people understood from the word go that it would never be the sole basis for a commercial product. And they were happy with that. Unfortunately, a part of Silicon Valley doesn't seem to understand this and companies keep cropping up that try to use open source to acquire customers.

Lots of companies have for many years understood the advantage of making some of their code open source (IBM and HP were traditionally known for this), not least because it absolved them of the responsibility of maintaining it. But, when it comes to your company's crown jewels or secret sauce, then don't open source them.

[Crypto Monad]

Re: Ugh...

> If you look back at the history of open source you'll generally see that people understood from the word go that it would never be the sole basis for a commercial product.

Not necessarily. The BSD licence is so permissive that it allows code to end up in proprietary, commercial products. This is by design: the people who work on BSD understand this very well.

[Michael Wojcik]

Re: Ugh...

Yes. Long before the FSF was a glint in Stallman's eye people were distributing software in source code and permitting its use in commercial products – regardless of whether it was the "sole basis".

And neither the FSF nor the OSI own the concept of open source or the term "open source", however they or their fans might wish otherwise. While various sources (including the inevitable one) credit Christine Petersen with coining it in 1998, "open source" was used at least in other contexts years earlier.

[Geez Money]

Re: Ugh...

Good reminder to never trust Wikipedia. That statement is wrong as hell. Wow. I personally first heard the term at least 5 years earlier. Pretty sure people used it to refer to MIT long before that. Isn't there a jargon file or c2 entry to disprove this nonsense?

[Jawn770]

It was a dick move, to be sure. Legal? Sure, but still deeply dick.

[Anonymous Coward]

Good luck to Elastic!

The claim that Open Source must be all or nothing doesn't click with me, given that end game for Amazon is leverage their size to crush Elastic anyway.

By crush I mean the Elastic will not be able to continue, because it is necessary to eat and have a roof in order to continue.

There's an Amazon employee who often comments on this issue - "Open Software writers are rich kids, real men make money. Screw Elastic".

Amazon's gonna do what they're gonna do, period. If Elastic didn't do what they are doing, they would be guilty of proving that Amazon employees forced viewpoint was the status quo we must accept and swallow.

[doublelayer]

"The claim that Open Source must be all or nothing doesn't click with me, given that end game for Amazon is leverage their size to crush Elastic anyway."

That's not nor was it their endgame. They wanted Elastic to keep making software so they could keep selling servers on which it's run. Killing them would do them no good at all. They contributed code for that purpose. Far from generous on their part, but your assumptions are entirely incorrect.

[Anonymous Coward]

So, they made a fork, have a few coders working on it, merging / porting changes that elastic do and are hoping all the contributors move to them so they don't have to put much work into it. Or if elastic dies and something, oh well, something else will show up doing the same job, so they don't have to much effort into it and if something doesn't, they still sell their server time, its not like the other providers are going to be supporting a dead project.

Less effort means less costs, which means more profit.

[doublelayer]

Yes, they are hoping to have others develop the code so they spend less. That's the nice part about open source--others sometimes do free work. Those others get, in return, a database they can use for free with a small set of known license restrictions. It's why we like open source. If it works right, everybody gets a good product they can use and modify without having to worry that someone will turn around and sue them for copyright violation, license fees, or similar.

[Anonymous Coward]

It's a mistake...

Don't most users use the AWS version of Elastic specifically because it is a managed service?

There's no way they want to run their own servers; using a 3rd party 'genuine Elastic' service within AWS will cost more; and using a 3rd party service outside of AWS will incur data exit fees so definitely cost more. Welcome to lock-in.

[The First Dave]

Re: It's a mistake...

How hard would it be, and how likely would it be, for AWS to add this header?

That was a common move in the old days of the browsers wars - every User Agent string said: " like that other browser" in order to deliberately break existing 'sniffing'.

[Anonymous Coward]

Good.

Amazon has been abusing the entire concept of open source for too long.

Sure, the original GPL didn't anticipate the abusive concept of "software as a service" - nobody did back then, because the entire concept of software depended on people running it on their own computers.

Elastic is doing the right thing. Amazon is not.

[doublelayer]

The idea of making money off a service while using others' open source code is not new. Software as a service was different at the time the GPL was drafted, but it wasn't unheard of. More importantly, stuff like that has been done in many other ways. One of the most obvious ways is shipping hardware with the open source code running on it. For example, anyone who manufactures computers running Linux is conceivably profiting from the operating system they installed. Yet we rarely see people calling organizations like the Raspberry Pi Foundation "abusing the entire concept of open source".

There is a good reason that's not what people say. They recognize that the use of open source software, in addition to being exactly what the license said you could do, also benefits the software community. The Raspberry Pi may make some money off their redistribution of Linux, but they also write code which benefits them and every other Linux user. In addition, their product adds more Linux users who will do the same. Compared to if the Raspberry Pi used something closed source, which would not get any of those benefits. That's why they are praised, not demonized. In AWS's case, the situation is similar--if Amazon built their own database, the Elastic project wouldn't get as many users, contributors, or donors. AWS also contributed code to the Elastic project. I get the feeling that Amazon had plenty of money and it would be nice for them to pay more of that to the rest of the Elastic devs. However, open source has never worked under the assumption that if you make money, you should be obligated to give that money to the developers. That is one of its strengths.

[Anonymous Coward]

Licences exist for a reason...

You need to select your licence to do what you want. That's why there are more than one.

Elastic Search chose to use Apache, that means that Amazon can do what they want with it. It's what the Apache license means. It's what the Apache license has *always* meant....

[FIA]

Re: Licences exist for a reason...

Yes. This!

I've said it time and time again.... if you only spend at least a small fraction of the time you'll spend developing something on considering what the licence you choose actually means then you'll be a lot happier in the long run.

My go to example would be WINE, it used to be released under the MIT licence, but this meant people took the hard work, forked it, and didn't release their improvements. Which, whilst exactly what the licence said they could do, turned out to be not what the project wanted people to do.

[Michael Wojcik]

Re: Licences exist for a reason...

And, conversely, some of us Really Don't Care what other people do with our code or other intellectual property, including selling it and forking it and adding proprietary elements to it.

[katrinab]

It was called a "time-sharing service" back then, but it was exactly the same as cloud computing now.

[Michael Wojcik]

Service bureaus are not "exactly the same" as cloud computing. They were a step in the direction of IT as a utility, but modern cloud platforms and infrastructure have gone far beyond what service bureaus did. And, importantly, the utility function for cloud computing is very different from that of service bureaus. The latter chiefly existed to amortize the large capital investment required for IT in various forms prior to 1990 or so, and so offered a value proposition mostly relevant to small and medium-size businesses. Cloud computing has a much flatter utility function with a much larger potential market because it's much more automated, has much finer granularity, and scales much larger.

And that's to be expected. Different economic constraints (and in this case they're very different on both the supply and consumption side) produce different economic formations. Disregarding those differences just produces false generalizations with little explanatory or predictive power.

Carr certainly wasn't right about everything in The Big Switch, but after 13 years his overall thesis is holding up pretty well.

I personally am not fond of cloud computing; I'm a hacker at heart and like to have control over the systems I work with. That's my comfort zone. And as an IT security professional I recognize the security issues of outsourced IT (though I'm also well aware of the issues with in-house IT...). But I understand the economics of cloud computing, how they beat the value propositions of earlier forms of centralized computing for many use cases, and why that's compelling to many organizations.

[Chronos]

TL;DR: All that is old shall become new. From a technical perspective, katrinab's comment is spot on, i.e. you do not fully control the hardware or software, you just transceive data. The rest is either sophistry or economics, things of which we tend not to take much notice on here unless it's the canteen onion bhaji budget.

TL as in you lost my interest at the word "amortize."

[Fruit and Nutcase]

Knickers

It is fortunate that Elastic, the company do not produce Elastic the material. If it did, then you may find your underwear without support whenever Elastic snaps.

[Michael Wojcik]

Re: Knickers

Well, I see my underwear has already been forked, so it's probably inevitable anyway.

[Anonymous Coward]

I don't see Elastic winning.

They may have changed the core Github code, but no one goes directly to that to grab Python libraries... Unless the distros want to get 1000 duplicate bugs "I updated the library and it stopped working with AWS", they'll need to patch this before shipping.

In any case, if Elastic are no longer happy with their choice of software license, they are allowed to change it to something else.

[Fruit and Nutcase]

Re: I don't see Elastic winning.

Reminds me of OpenOffice v LibreOffice

[Charlie Clark]

Re: I don't see Elastic winning.

If it hasn't already happened, this library will be forked and probably quickly become the standard one, because AWS dominates when it comes to use. Then Elastic will have all the work of backporting, integrating, disabling, etc. to support its own customers.

[katrinab]

Re: I don't see Elastic winning.

On Windows, I think most people would get the library from pip or conda, and I guess they would get it from GitHub. Windows probably doesn't have a huge market share in that segment, but presumably there are some people that use it.

[Anonymous Coward]

So forks just need to send out the correct "X-Elastic-Product" header, but also send their own X-Elastic-Real-Product. Problem solved. Arms race started.

[pip25]

Elastic is deluded

I cannot think of any other reason why they'd believe that this license change and client shenanigans aren't going to blow up in their faces.

Not to say that OpenSearch is guaranteed to win. The most likely scenario is that everyone will lose. :(

[Falmari]

Spirit of opensource

I have very little experience with opensource Elasticsearch the one I have had most with. The products I work on only use two pieces of opensource one is used as a library, we compile the source into our codebase. The second is Elasticsearch, we make API calls through .Net to the customer’s installation of Elasticsearch.

Therefore, I would consider myself and outsider to opensource and the community. But to these outsider’s eyes the actions of both AWS and Elastic though within the rules go against the spirit of opensource. To me they run contrary to the founding ideals of open source.

I did the PoC and prototype for our implementation of Elasticsearch, that’s where my experience ends another team implemented the feature. But before I did the PoC I went on a course run by Elastic. In one lunch time break I got talking to the presenters one a feature writer the other a VP of something or other.

I asked them how Elastic can make money when they have open sourced it? They said it was through licensed support, training courses and the tools and extra features they sell to go with it.

AWS’s cloudy implementation of Elasticsearch does not licence Elastic’s tools, AWS created their own. So even though the license allows AWS to do this, to Elastic AWS depriving Elastic of their revenue stream.

That said Elastics actions in changing the licence on the opensource parts goes against the spirit of opensource, Elastic though the main contributors are not the only contributors. Contributors who submitted under one licence have had their contributions re-licensed.

Both AWS’s and Elastic’s actions to me go against the spirit of opensource.

PS Yes, I know I have written so much to say so little. :)