Open source registries don't have enough money to implement basic security

Open source registries are in financial peril, a co-founder of an open source security foundation warned after inspecting their books. And it's not just the bandwidth costs that are killing them. "The problem is they don't have enough money to spend on the very security features that we all desperately need to stop being a …

[xcdb]

Unquestioning, much?

"In some cases benevolent parties can cover these bills: Python's PyPI registry bandwidth needs for shipping copies of its 700,000+ packages (amounting to 747PB annually at a sustained rate of 189 Gbps) are underwritten by Fastly, for instance. Otherwise, the project would have to pony up about $1.8 million a month."

If you were insane enough to pay AWS list prices, but who at that scale would be doing that?

A quick look at OVH suggests that unmetered 25Gbps is about £1,000/month, so even if you were bonkers and didn't cache (and ignoring hosting fees and using only the sustained figure of ~200Gbps) it would be about $11,000/month for the bandwidth vs the article "~$1,800,000/month"

[xcdb]

Re: Unquestioning, much?

Well, I wasn't trying to do a detailed analysis more of a "nah, that can't be right" and taking the article claim of "189 Gbps" spent 5 mins seeing what a similar aggregate bandwidth could cost...

[dgeb]

Re: Unquestioning, much?

Cloudfront bandwidth pricing for it works out like this, per https://aws.amazon.com/cloudfront/pricing/pay-as-you-go/:

747PB/yr = 62.25 PB/mo

1TB @ $0.00 = $0

9TB @ $0.085/GB = $765

40TB @ $0.08/GB = $3,200

100TB @ $0.06/GB = $6,000

350TB @ $0.04/GB = $14,000

524TB @ $0.03/GB = $15,720

4PB @ $0.025/GB = $100,000

57.25PB @ $0.02/GB = $1,145,000

Total = $1,284,685

(This is based on it all being in either North America or Europe, which have the joint lowest rates - the blended cost for global traffic will be somewhat higher, although we lack sufficient data to know by how much.)

So far from being around 2-3% of list price, it's very much in the ballpark. Although very likely you're right that the $1.8m was a Fastly figure, they only publish low-volume pricing so we can't check it.

[Jeff 11]

Re: Unquestioning, much?

In a like-for-like cost of doing it yourself, that figure is going to be inflated by the CDN providing lots of edge-side replication and more peak capacity - 189Gbps is only an average. But indeed not to the tune of 100-200x what you've calculated...

[Charlie Clark]

Re: Unquestioning, much?

Something the article doesn't mention is that most of the traffic is related to CI setups. Python experienced this first when easy_install and later buildout were released 20-odd years ago. This made reproducible installs a lot easier and CI started popping up a few years later, which increased the load ten-fold.

Then again the rise of widescale DVCS and CI has arguably made software a lot better for users but caching could be improved.

[alain williams]

Name and Shame ?

Some very large & very profitable enterprises are heavy users of open source but contribute nothing to what they depend on.

if projects were to name the non-paying skinflint companies they might be able to shame them into making contributions.

[VoiceOfTruth]

Re: Name and Shame ?

That old chestnut again - make the 'big' users pay.

If you use open source, you are a user too. Reach into your pocket.

[alain williams]

Re: Name and Shame ?

Reach into your pocket.

I do.

Do you ?

[VoiceOfTruth]

Re: Name and Shame ?

Votes down are presumably from people who use open source, and want other people to pay for its development.

[chuckufarley]

Re: Name and Shame ?

I down voted and I want to pay pay. I'm just poor and can't afford medical services let alone software.

[Blazde]

Re: Name and Shame ?

We all use open-source, directly or indirectly. It's well past the point central governments should acknowledge this. A quick working session between finance deputies at the next G7 could have it solved using insignificant amounts of general taxation, followed by an all-smiles bland announcement about 'committing to growth and security in a rapidly changing and challenged world' or some such.

Abdicating this responsibility to big tech companies effectively only hands them more power they don't need.

[coredump]

Re: Name and Shame ?

Some of those 'big' users are making a (big) profit from others' open source. They're in a much better position to give back to the open source project and developers than your average Linux or BSD hobbyist working on a 2nd-hand decade-old laptop.

Granted, some of them do: e.g. hiring devs, donating to the project (or its foundation) and so on. And that's great. The funds, as well as setting a good example.

I'm not saying every big corporation that runs some Linux in-house is obligated to tithe 20% to Debian or OpenBSD (everybody uses SSH, right?) etc., but some of those outfits are pocketing enormous profits, and the rounding error from their quarterly spreadsheet could make a dramatic difference to some open source projects; especially the ones not backed directly by a corporation of their own.

Plus, giving back to those projects probably helps the company, even indirectly. You'd hope they could be motivated by a little self-interest now and then.

[VoiceOfTruth]

Re: Name and Shame ?

Boo hoo. If you use open source, stop being a hypocrite and contribute back. Some people want everything for free = somebody else to pay and give their time and money,

[coredump]

Re: Name and Shame ?

I do. Do you?

Nevermind. I don't really care whether you do or not.

Your accusation of hypocrisy is inaccurate and insulting. Your "boo hoo" is non-sequitur nonsense. Whether I (or you) give, or not, doesn't relieve others like big corporations of any responsibility for giving back also.

[werdsmith]

Re: Name and Shame ?

If you are a hobbyist, self-training or developing something new or demonstrating your skills then the free part benefits everyone.

If you are using someone else’s voluntary effort to line your own pockets then I suggest that you owe someone a debt.

[Irongut]

> Hospitals, universities, and museums are all nonprofits, yet they still charge for services.

No, that very much depends on where you live as I have never been charged by a hospital or museum for their services.

[alain williams]

Hospital charges

No, that very much depends on where you live as I have never been charged by a hospital or museum for their services.

Have you been in a hospital car park recently ? I was charged £4 for ½ hour at the Lister hospital (Stevenage) recently :-(

[Noodle]

Re: Hospital charges

It's unlikely you are paying a hospital for parking though, probably a third party operating a car park on hospital property. Unless you consider car parking to be a service that hospitals should offer but personally I'm happy with them focusing on medical care.

[AndrueC]

Re: Hospital charges

Parking is free at Welsh Hospitals.

You'll struggle to find a space, mind, but it's free.

[werdsmith]

Re: Hospital charges

Alain, park for free in the Sainsbury’s and do the 2 minute walk. You can walk further within that hospital anyway.

And anyway, the nearer on street parking is £3.60 for 2 hours.

You are being charged for car use, not hospital services.

[Tron]

Stuff costs money.

I spent most of 2025 in and out of NHS hospitals. Overall that was about £1000 in costs for taxis and prescription charges. I guess you must live next to a hospital and round the corner from a museum.

[Anonymous Coward]

Re: Stuff costs money.

I don't, I live ~30 miles from the hospital, cost me nothing to park at the hospital when being a patient. When not one, cost around 20p / hour. There is the fuel costs, but that nothing to do with them, that's me and my mode of transport that determines those costs.

But i no longer live in the UK.

[sabroni]

re: I have never been charged by a hospital or museum for their services.

And back when I was a kid you were entitled to an education for nothing too. Almost like we understood that educating people was a benefit to society in general.

[doublelayer]

I get your point, but it's wrong from the perspective of the comment you're talking about. Hospitals still charged for the services you received. They didn't charge it directly to you. The funds from taxation paid that bill. If they hadn't charged, then hospitals would be operating with only voluntary and donated resources, but that's not how they work whether you are in a country where the patient pays when they use it or the government pays more generally.

Museums sometimes do operate that way, though many I've attended were not government run so did charge for entry, were government-run but still charged fees for entry, or received funding from a different source which itself collected fees. The UK is no exception to that, with the National Trust being that source for quite a few places.

[Claptrap314]

The other half of the problem

when one speaks of security an OS repository, it is critical to answer "for whom"--because there is actually a substantial mismatch between security for providers and security for users.

OS providers need to be able to control what is in the repo under their name. For them, it's about identity and the ability to push and even delete code. Users certain care about identity, but when a dev deletes code, that can be has been a problem.

A really NASTY problem happens when there is a semver failure deep in the stack. (We had a patch pushed to a small gem that broke a popular dependency. It took months for new versions of all of the dependencies to push fixes.)

From an availability (and security) standpoint, end users must maintain their own versions of the repos. Versions that never delete anything, and from which build systems exclusively pull code. Copies that can be manipulated at will if necessary.

[ChoHag]

How about open sores steps up and provides the people and bloated corporations (but I repeat myself...) who are using the free software (which, you will recall, is provided "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED) with all the security guarantees and massively online mirrors they're paying for? It's only fair.

[chuckufarley]

Just what the retiries need...

...having to pay for something that sets them free because it is free. Yes, the registries need a helping hand but asking everyone to pay isn't going to work when the the organizations that gain the top 60% of the befits could foot 100% of the bill while losing less then 0.01% of their profits and the people they abuse everyday must resort to public assistance just to eat. Remember that dying of malnutrition is a lot like dying of starvation but it takes years, if not decades, longer.

[Eric 9001]

If you're centralizing package downloads that much

You're doing something very wrong.

I'm aware of several free software mirrors that are distributed and work fine at the cost of residential internet connections (that somehow get better uptime than "cloud" garbage).

Malware isn't a problem if it's made sure to check that a new developer is in fact a new developer is into freedom and are uploading free software before giving them direct upload access (the xz-utils attacker(s) had to contribute free software for like 1.5 years before they were given the ability to make xz-utils releases and the proprietary systemd/Linux backdoor, ended up being caught pretty much immediately).

It seems proprietary malware is only a big problem if the software repository doesn't have the policy that only free software is allowed and any and all proprietary software will be removed when reported.

[O'Reg Inalsin]

Security first - don't promise what cannot be delivered

High load registries should just charge a subscription. Downloading the code from the published source and compiling/building (or whatever) is always an option.

[Eric 9001]

Re: Security first - don't promise what cannot be delivered

Okay, but how do they accept payment?

If they go with credit card, that generally means requiring the execution of proprietary software to get free software? (clearly few would go with a subscription).

Also, it's generally cheaper to copy software and/or compile yourself exactly to your needs, than to pay someone else to do so (who usually won't match your needs exactly and thus have extra costs).

[Noodle]

"yet they still charge for services. In fact it is good practice; otherwise people will abuse the system"

Tell me you're an American without telling me you're American.

[saltycupcakes]

The solution is pretty obvious

Rate limiting doesn't really impact hobbyists or open source devs, I mean who downloads 100 docker containers in a single month let alone single day, but it forces enterprises to either cache images, reducing the costs for registries, or makes them pay for a docker license, giving them the income they need, its a no brainer really.

[doublelayer]

Re: The solution is pretty obvious

That depends how you rate limit. The easy option is to limit by IP address. No need to identify every user, track accounts across the servers that distribute files, or any of that. When someone buys a license, they can give you addresses and increase the limit on those. What's not to like? The problem is that businesses may have a few hundred or a few hundred thousand addresses, so they can distribute requests over those if they like, while some people are shoved into multilayer CGNAT and will never be able to escape the limits. For businesses that don't work around it, it works, because they'll often make most requests from a few static office/VPN endpoint addresses and from a few build servers, but that's no guarantee they won't find a way around it.

[Christian Berger]

Well the problem is unneeded dependencies

Solutions like PyPi make it seem like having dependencies is easy and no problem, by hiding the problems. This makes people use more and more dependencies, making the problem much worse.

That's why I try to avoid them. If I write software in Python and a library is not available in Debian/Ubuntu, I will not use it.