The quest to make Linux bulletproof

This is the second half of a feature about work undertaken to harden and improve Linux, beginning with part 1 here. Commercial Unix was expensive so it was carefully tended – and indeed tendered. Linux is free so it has to fend for itself. Linux itself was inspired by the tried and tested designs of the proprietary Unixes …

[Anonymous Coward]

Optimism can be a real problem!!!

Quote: "It just needs us to let go of the idea of human-managed file systems, and let the computer handle it."

That's fine and dandy.......until something in "the computer" goes seriously wrong. That's when the humans who "let go" find they have a serious problem.

Perhaps you remember this type of approach being taken in the design of the Boeing 737 Max?

[Anonymous Coward]

Re: Optimism can be a real problem!!!

Hello:

The sysadmin must let go of the idea of knowing what is kept where, of being able to read and understand folder names ...

Yes?

You don't say?

What a really interesting * idea.

* ie: absolute bullshit.

... until something in "the computer" goes seriously wrong. That's when the humans who "let go" find they have a serious problem.

+1

+10 ...

Indeed.

I logged in to say the exact same thing.

[JoeCool]

That's a management problem.

You have to choose an admin : either the human OR the flatpack.

What I wonder is how the flatpack will order and install the additional storage it needs. Did a corporate credit card get registered at install time ?

[yetanotheraoc]

let go of the idea of knowing what is kept where

It's just another layer of abstraction. It doesn't do away with the complexity, it just holds the struggling body completely underwater so the surface appears smooth.

(A) *Somebody* still needs to know where the file went. All file systems are in fact databases, that's not radical. The directory path is one possible conceptual model on top of the database, it just happens to be an easy one for humans to wrap their heads around. You say the sysadmin doesn't need it? Fine, but how will the sysadmin know the fetched file is today's rather than yesterday's, or indeed some out-of-tree version?

(B) Frameworks only allow for (most) common use cases. If you have a special case, you will have to drop down to the lower level. Does the abstraction in question even *allow* that? When I use an admin GUI, it's just a convenient way of issuing commands or modifying a config file. That helps me if I want it, but also gets out of my way if I don't want it. Similar for some IDE sitting on top of a compiled programming language. If I don't want to use an IDE, I can code with ed if that floats my boat.

(C) What happens when something goes wrong? Can the intrepid sysadmin attach a debugger and step through, or is it bug reports all the way? When every bug report is met by "cannot reproduce" [0], e.g. blame the user's config (that's especially pernicious when the system chose all the config on the user's behalf) ; otherwise met by "won't fix", e.g. because fixing one bug would break another use case; *then* we know the abstraction has gone one level too far.

[0] Oh, here's an idea -- telemetry!

[Anonymous Coward]

Re: let go of the idea of knowing what is kept where

It doesn't do away with the complexity, it just holds the struggling body completely underwater so the surface appears smooth.

Both suitably dark and 100% perfectly expressed. Respect, I'm going to have to borrow this one. A lot :).

[JoeCool]

That's a reach !

The description reminded me much more of Make :

Programmer writes makefile which specs WHAT is supposed to get built,

Make decides HOW to do it,

and when it fails, the Programmer needs to RTFSource to get a working build (and on proprietary Unix THERE IS NO SOURCE).

</flashbackhorror>

[Ignazio]

Re: Optimism can be a real problem!!!

Good question and wrong example. The max failure was due to changing some important things without changing many other important things, not least of which being 'pilots need retraining' and 'poorly tested software attempting to cover for massive hardware changes, some of which made the plane impossible to handle' /he says, simplifying things quite a bit.

Whereas here the change isn't that far reaching and the nature of the beast hasn't mutated. DLL Hell, dependency hell, wasted space or slow start are the worst case scenarios, and, well, they're all problems that exist in the current state of things, at compile time or runtime.

So, optimism is allowable.

[EricB123]

Re: Optimism can be a real problem!!!

Excellent observation about the 737 Max. May I add that the new efficient engines were too big for the aging 737 design, and the engines would have scraped the runway so they moved them forward a bit. This made the plane susceptible to stalls, so they added that dreaded software to "fix" the problem.

[Anonymous Coward]

Re: Optimism can be a real problem!!!

And then the Pakled... will need to find an engineer to make IT go

[Doctor Syntax]

"The only way to install software, including updates, is during a reboot, using a new command,"

This sounds strangely familiar.

Oh what a tangled web we weave when we...abandon KISS

[b0llchit]

KISS is eaten by ACID.

You need serious BASE to add to ACID for neutrality. You also need more BASE when you want more ACID. In this CASE there will be fewer KISSes allowed and everything needs to be locked into the CASEs for isolation. Therefore, love is in serious jeopardy.

^{FWIW, I'd rather have more KISSes than racing against the ACIDs and BASEs in the CASEs.}

[chasil]

Columbus UNIX

This assertion by the author is not correct:

"In the Linux world, this first appeared with journaling file systems."

What is actually needed for databases is System V IPC - semaphores, message queues, and shared memory.

Columbus UNIX introduced these, long before journaling file systems.

https://en.wikipedia.org/wiki/CB_UNIX

"CB UNIX was developed to address deficiencies inherent in Research Unix, notably the lack of interprocess communication (IPC) and file locking, considered essential for a database management system... The interprocess communication features developed for CB UNIX were message queues, semaphores and shared memory support. These eventually appeared in mainstream Unix systems starting with System V in 1983, and are now collectively known as System V IPC."

[that one in the corner]

Re: Columbus UNIX

> What is actually needed for databases is System V IPC - semaphores, message queues, and shared memory

Well, not quite. IPC is only *needed* if you have more than one process on the machine (can I get a "duh"?) and they need to safely and efficiently communicate with each other (as with the term "IPC" itself we're using the old meaning of "process" and avoiding arguing over terms like "thread").

"A database" with all the ACID you could want can be run inside a single process. There is nothing fundamental in the concept of "database" that needs anything more. You can even run it as a client/server setup - many ways exist, but just having a few serial lines and poll them every now and again will work.

Of course, even on a single CPU box you're going to find that method a pain in the arse to use, especially when you start whacking on pre-emptive multitasking because, guess what, you've just implemented processes and *any* robust & useful system with processes needs IPC. Databases just being one workload that can be made more efficient with IPC (e.g. we're running local clients now and is a bit wasteful to use up two serial ports on one machine with our original implementation).

IPC is definitely required by the time you are running your database engine across multiple CPUs in one box, otherwise it'll all get very nasty!

So for big databases on big machines, yup, you want your IPC. For big *anything* on big machines you'll want your IPC.

HOWEVER leaving all that aside, the line you are objecting to was saying that the ACID principles that were applied to the problem of OS updates were those that were built into the journalling filesystems - because no-one (we're aware of) had tried to put all of the OS files into a.n other "more obviously a database". If they had, presumably that'd be replacing the "dir" command by an SQL query? Otherwise, if it exposed the normal FS operations so "dir" (oh, alright, "ls") worked then you've just invented the journalling filesystem and we're right back to agreeing with Liam's original sentence!

[Michael Wojcik]

Re: Columbus UNIX

IPC is only *needed* if you have more than one process on the machine (can I get a "duh"?) and they need to safely and efficiently communicate with each other

Not even then, necessarily. On UNIX, if the processes are related, they can communicate using pipes, and obviously those were available in Research UNIX prior to Columbus. Even unrelated processes can using FIFOs, which I believe predated Columbus IPCs in AT&T UNIX. (Research UNIX didn't get BSDisms, including socketpairs and UNIX-domain sockets, until 1985, so Columbus IPC predates anything socket-related in the AT&T UNIX family, I believe. But a couple of FIFOs give you most of the same capabilities.)

Communication through pipes or FIFOs isn't exactly fast; it's doing byte-wise I/O with a trip through the kernel, so there are two context switches and at least two copies. It's certainly slower than shared memory, even with the necessary synchronization. But a DBMS of that era is going to be doing a lot of disk I/O, given the cost of RAM and limitation of address space, and that's going to be far slower.

So Columbus IPC was useful for DBMSes but not a requirement. INGRES was running on BSD without any "modern" IPC mechanisms back in the 1970s, if memory serves.

[Anonymous Coward]

"I'm sorry Dave, I can't do that."

See title!!!

[Anonymous Coward]

Reading this makes me tired...

My take home from all this is that Linux is still a complete mess of competing standards, versions, implementations, flavours, whatever - and that it and the community supporting it just don't seem to get that creating yet another version to solve the problem just... adds to the problem. It may be an acceptable situation for professional greybeards who have *their* systems running *their* preferred setup, but as an ecosystem its awful.

[Anonymous Coward]

Re: Reading this makes me tired...

I've been running Linux and macOS both for years. macOS is from a single vendor, they control it, it's consistent-ish and there's just one standard, but when it goes wrong I'm stuffed. Centralised control means I don't have the knowledge or tools to fix it, and things don't evolve: the first thing I install on a new Mac is brew (which gives access to modern versions of the outdated dev tools shipped with macOS). And bugs I reported and sometimes even diagnosed in macOS Preview.app and Terminal.app still aren't fixed, after years.

Linux absolutely is a mess of competing standards, but that competition means that tools have to place nicely with other tools, they're documented and that the cream rises to the top. Not always, but often enough. I've gone from ext2 to ext3+lvm to XFS+lvm to ZFS, each an improvement. cpio gave way to tar.gz, and now apt, also an improvement. I wouldn't say it's Darwinian but ecosystems need a certain amount of chaos to evolve.

[Rich 2]

Re: Reading this makes me tired...

Unfortunately, some of what you say is simply not true

A lot of Linux tools are appallingly badly documented and quite a few really don’t play nicely - I’m thinking systend. As for the crude oil rising to the top…

And BSD doesn’t seem to suffer these issues, and that runs on many platforms (unlike OSX)

[Gene Cash]

Re: Reading this makes me tired...

I’m thinking systemd

Well then just bloody don't use systemd! That's what I did. I left Debian for Devuan and I no longer use RedHat.

That's the guy's point. Enough people stop using the crap tool, and use something better.

[Rich 2]

Re: Reading this makes me tired...

I don’t use systemd

….Obviously

[Anonymous Coward]

Re: the cream rises to the top

You think that brown floater is cream you go right on and gobble it up!

[JoeCool]

Nope. This is what bleeding edge innovation looks like.

It's a market place gold rush.

[currynut]

Re: Reading this makes me tired...

It's not unified for sure - but is it a mess? You seem to be referring to opinionated distro/tool fanatics and public debate (not exclusive to Linux). At the same time Linux servers (mainstream Debian/RHEL flavoured) support "a lot of the internet" (*) while nobody (including the on-the-hook sysadmins) cares. A contrast with what _isn't_ a mess.. mainframe z/OS, Windows?.. would be helpful to understand your point.

If you are referring to Linux on the desktop issue then I'll concede that I don't give a sh**.

(*) not fact checked - but most likely true.

[elsergiovolador]

Blockchain

You just need to have blockchain backed filesystem.

Imagine being able to detect that someone (likely yourself when you meet walrus after taking too much Ambien after long day at work) tampered with your precious files.

[Ignazio]

Re: Blockchain

Sarcasm, right?

[that one in the corner]

Re: Blockchain

Well, if not, can we put together a project to get voting control over his blockchain, make it believe that his box has always been running WindowsME, so the next update that tries to install a patch fails, triggering a rollback to last known good?

[Brewster's Angle Grinder]

Remember when BillyG was going to replace the filesystem with a database? If Liam's closing thesis is right, then that's the direction we're heading.

It all makes me pine for the days of RiscOS when everything the app needed was in it's folder and spreading yourself across the OS was rude.

[phuzz]

Alas poor WinFS, we never knew ye

[Alan J. Wylie]

replace the filesystem with a database

See ICL's Content Addressable File Store

the search logic was incorporated into the disk controller. A query expressed in a high-level query language could be compiled into a search specification that was then sent to the disk controller for execution. Initially this capability was integrated into ICL's own Querymaster query language, which worked in conjunction with the IDMS database; subsequently it was integrated into the ICL VME port of the Ingres relational database.

[captain veg]

replace the filesystem with a database

See also Pick.

-A.

[Gene Cash]

Re: replace the filesystem with a database

or BeOS

[James Loughner]

"BillyG was going to replace the filesystem with a database"

BTRFS => B-tree file system. B-trees are databases Even if M$ could not do it LOL

[Brewster's Angle Grinder]

If B-trees are databases, my apps are full of databases - none of which look anything like a conventional database you'd recognise.

[Dickie Mosfet]

"It all makes me pine for the days of RiscOS when everything the app needed was in it's folder and spreading yourself across the OS was rude."

[GoboLinux has entered the chat]

https://gobolinux.org/

[TVU]

The quest to make Linux bulletproof

I just wanted to thank the author for differentiating between the OpenZFS file system and proprietary Oracle ZFS and I would like to see more of that happening.

There is still the potential for Oracle to initiate vexatious court case litigation which is why support for OpenZFS is not currently included in the Linux kernel. Understandably, Linus Torvalds does not want to include such support in the kernel until a categorical assurance is given:

"And honestly, there is no way I can merge any of the ZFS efforts until I get an official letter from Oracle that is signed by their main legal counsel or preferably by Larry Ellison himself that says that yes, it’s ok to do so and treat the end result as GPL’d".

[Bebu]

Don't give up the day job

"git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space."

Yes apparently a joke

"https://softwareengineering.stackexchange.com/questions/256450/are-git-branches-in-fact-homeomorphic-endofunctors-mapping-submanifolds-of-a"

and codswallop. The Haskell monad/monoid thing it parodies is apparently correct and humorous.

Comedy night in the maths dept must be a barrel of laughs.

[LionelB]

Re: Don't give up the day job

Infinitely many mathematicians walk into a bar. The first says, "I'll have a beer." The second says, "I'll have half a beer." The third says, "I'll have a quarter of a beer." And so on. The barman pulls out just two beers. The mathematicians are all like, "That's all you're giving us? How drunk do you expect us to get on that?" The bartender says, "Come on guys. Know your limits."

Infinitely many mathematicians walk into a bar. The first says, "I'll have a beer." The second says, "I'll have half a beer." The third says, "I'll have a third of a beer." And so on. The bartender shouts, "Get the hell out of here, are you trying to ruin me?"

[Anonymous Coward]

...And Other Conundrums.......

@LionelB

The Infinity Hotel is completely full.

A tour group arrives -- with an infinite number of touroids in the group! What to do?

The hotel manager sends a message to all the people already in the hotel: "Please move to another room -- just multiply your room number by two. So...room 1 to room 2....room 101 to room 202..."

And now the incoming infinite tour group gets assigned to rooms with odd numbers: 1, 3, 5.....101, 103, 105....

In fact, when the the Infinity Hotel is full, it can still accomodate an infinite number of infinitely large NEWLY ARRIVED tour groups!!!

Ref: "What is the name of this book?", Raymond M. Smullyan

[yetanotheraoc]

Re: ...And Other Conundrums.......

"In fact, when the the Infinity Hotel is full, it can still accomodate an infinite number of infinitely large NEWLY ARRIVED tour groups!!!"

On the minus side, it takes a new tour group an infinitely long time to check in, and the one after that takes a longer infinity. Don't even think about calling the front desk complaining there's no hot water. They're busy.

[Claptrap314]

Re: ...And Other Conundrums.......

Nope. The Hilbert Hotel is countably infinite, so all the infinities are the same here.

[yetanotheraoc]

Re: ...And Other Conundrums.......

It's an interesting thought experiment, but despite havng his name on the sign it's clear Mr. Hilbert never worked a day at the front desk. It's all well and good to say infinite bus 1 is the same size as infinite bus 2 is the same size as their sum. The fact remains when infinite bus 2 arrives they need to get in line, and since infinite bus 1 takes infinite time to finish checking in, the passengers on infinite bus 2 better have snacks with them. As for an infinite number of infinite buses, I heard they are stuck in traffic.

[LionelB]

Re: ...And Other Conundrums.......

> It's an interesting thought experiment, but despite havng his name on the sign it's clear Mr. Hilbert never worked a day at the front desk.

No problem; the check-in clerk is super-efficient. They can always check a new guest in in half the time they took to check in the previous guest.

[FelixReg]

Re: ...And Other Conundrums.......

Well now, bus 2 doesn't need to get in line after bus 1. Instead the clerk checks in a guest from each of the buses before checking in the next guest from each bus. Now, if there are an unlimited number of buses? Perhaps the clerk could take guests from 1 bus, first. Then guests from 2 buses. 3 buses, and so on.

[Claptrap314]

Re: ...And Other Conundrums.......

Just to clarify for those who haven't played before:

Step 1/1: Take person 1 from bus 1.

Step 2/1: Take person 1 from bus 2.

Step 1/2: Take person 2 from bus 1.

Step 3/1: Take person 1 from bus 3.

Step 2/2: Take person 2 from bus 2.

Step 1/3: Take person 3 from bus 1.

...

And yes, that clerk learns. Each client is checked in in half the time of the one before.

[Will Godfrey]

Hmmm

Seeing as Linus is not British, it's a bit pointless referring to a British slang.

I'm no expert by any means, but Git has got me out of far more trouble than it's got me in to - especially when trying to find one of my less smart changes.

[JoeCool]

Re: Hmmm

Is Linus's Git pronounced differently ?

[Justicesays]

Re: Hmmm

Isn't is pronounced "Jit" ?

[John H Woods]

Re: Hmmm

Oh for Jod's sake...

[LionelB]

Re: Hmmm

No, the guy's Finnish. It's clearly "Geet".

[jake]

Re: Hmmm

Relax. Many of us Yanks use the term, too.

Note that Linus says he's an an egotistical bastard and names all his projects after himself ... although that claim was made long after the fact. Those of us in the trenches at the time know who it was really named after.

[sabroni]

Re: Those of us in the trenches at the time know who it was really named after.

Was it a tedious twat who couldn't stop harping on about being around at the birth of the internet? ;-P

[jake]

Re: Those of us in the trenches at the time know who it was really named after.

No, Larry's not quite old enough for that. He's also not prone to make snide non-sequitur comments in obscure internet forums.

HTH, HAND

[Børge Nøst]

New approaches - ame direction

It seems like all of these are still basing themselves on a reboot for upgrading. I thought the Linux world had progressed further in hotpatching.

[theOtherJT]

It's all about making it easier...

...for humans to manage the huge, sprawling complexity of modern apps

One can't help but wonder if it might actually make sense to go and look at said "modern" apps and determine why they got so huge, sprawling, and complex - because I'm almost certain that in 90% of cases what we're doing here is building complexity in order to manage complexity, rather than doing the smart thing and trying to take the initial complexity out.

I'm sure there are plenty of greyer heads than mine in this here comments section, but I've been doing this quite a while now and I'm far from convinced that most of what we do today is objectively better than what we were doing when I joined the industry twenty years ago. It's more just sort of... different... and probably involves a web browser for some reason no one seems to be able to adequately explain.

[MrBanana]

Re: It's all about making it easier...

We all know. Sprawling, unnecessarily complex... Go on, just say systemd.

[Anonymous Coward]

Re: It's all about making it easier...

systemd is a great example but far from the only one...

[jake]

Re: It's all about making it easier...

There is always EMACS ...

::ducks::

[Fifth Horseman]

Re: It's all about making it easier...

Absolutely.

"Simplicate, and add lightness" - William Bushnell Stout. Adopted by Colin Chapman as the design ethos for Lotus sports cars.

[drankinatty]

Re: It's all about making it easier...

... making the root file system read-only. The only way to install software, including updates, is during a reboot, using a new command, transactional-update... WTF?? Easier, you have got to be kidding me???

I just needed the update to dos2unix 7.4.4.1 -- and now I have to reboot? No thanks.

...If you have a cluster of hosts running lots of containers, this should not be too intrusive ... It's less convenient for a non-clustered machine ... (ya think?)

How many of you are reading this off a cluster? SUSE's ALP will be a cluster alright for the community of users that have supported it for the last two decades, a cluster ....

Seems distributions that once championed "User Choice" in Linux are now abandoning the community and choice in favor of the corporate customer.

[jlturriff]

Re: It's all about making it easier...

"Seems distributions that once championed "User Choice" in Linux are now abandoning the community and choice in favor of the corporate customer."

Indeed.

And from desktop environments (Gnome, KDE) to init -> systemd, developers seem to have forgotten in many cases that there are people (both individual and corporate) who use Linux for real work, who need stability to get their jobs done, in favour of the newest "gee-whiz" conceptual "features," many of which have not been requested by the end-users of Linux. The corporate users have the time and resources to retrain their staff to adapt to these gratuitous changes, but not without considerable cost; and private users are left in the dark about what will happen to their systems as the changes are implemented. Gnome and KDE continue to make radical changes to their desktop environments; OpenSUSE surprised us with BtrFS, systemd, and stuff like PulseAudio and the like, and now we hear that Leap will be discontinued, that everything will move to package virtualization, but we don't need to know the details because 'everything will be lovely, lovely by and by...'

[sabroni]

Re: probably involves a web browser for some reason no one seems to be able to adequately explain.

I'm just talking off the cuff here, so I might be wildly wrong, but I thought the reason we wanted to run in the browser was because it means we don't have to install anything on the client pc?

You've been doing this for 20 years and not worked that out?

[yetanotheraoc]

Re: probably involves a web browser for some reason no one seems to be able to adequately explain.

"because it means we don't have to install anything on the client pc?"

If by not install you mean we pull down a shit-ton of javascript, then yes, that's exactly why they (not we) want to use a web browser.

[doublelayer]

Re: probably involves a web browser for some reason no one seems to be able to adequately explain.

No, by "not install" they mean to have no persistence. Pulling down a bunch of JS can, if it doesn't use some of the new filesystem APIs that are in browsers for some reason it's better not to think about, run an application while not having any permanent effect on the system after the application closes. There are times when that is desirable. There are a lot of times where it isn't, which is useful to keep in mind, but not every web-based application would be better as a client program.

[Anonymous Coward]

Re: It's all about making it easier...

> ... and probably involves a web browser for some reason no one seems to be able to adequately explain.

Putting stuff in a browser is what they do these days rather than writing a separate GUI.

There seems to be a prevailing feeling that if a command or tool "looks pretty", has a form full of blanks to fill in before pressing [Go], and a little "?" hover button for more info, that's somehow "better".

It was one thing when you had to have X[11,org,free86] running to do any of that; now you need to have a webserver too.

I fear someday there won't be man pages and commands won't take "-h|--help" flags anymore.

[Anonymous Coward]

For such a supposedly scholarly article

(by length leastways)

not a single mention of puppet is an odd omission.

[Alan J. Wylie]

Re: For such a supposedly scholarly article

not a single mention of puppet

Perhaps Liam didn't want to trigger anyone's nightmarish flashbacks.

[Liam Proven]

Re: For such a supposedly scholarly article

[Author here]

> (by length leastways)

«Je n’ai fait celle-ci plus longue que parce que je n’ai pas eu le loisir de la faire plus courte.»

> not a single mention of puppet is an odd omission.

Is there a single Linux distro that has Puppet built-in? I don't think I know of one. It's an external tool, and it's orthogonal to all of this. (Well, except Nix, which could replace that, as well.)

And if I went into that, I'd need to talk about Chef, and Ansible, and Salt, and it would be much longer still.

I drew a line at tools which are part of the design of different distros, and I think it was a reasonable one to draw.

[rafff]

Snap is a single compressed file,

So IIUC dynamically linked libraries are, to all intents and purposes, a thing of the past: each snap has its own copy of every library it uses.

[Liam Proven]

Re: Snap is a single compressed file,

[Author here]

> each snap has its own copy of every library it uses.

Broadly, yes. There are dependency mechanisms in both Flatpak and Snap that mean that the system has "a GNOME snap" and "a KDE snap" and all the apps for that desktop depend on it, so they don't need to install the whole thing.

But the idea is that if one app needs one specific version of a library, then you don't break it by updating that library to a newer one. Yes, it is wasteful of space, but then space is cheap these days. A 256GB µSD for your Raspberry Pi is about $/€/£ 20 now. 512GB is under $/€/£ 50. Even embedded SBCs can afford to burn through a few extra gigs if it makes them more reliable and robust.

[runt row raggy]

Re: Snap is a single compressed file,

the reason you don't want different libraries for each app is then n copies of libraries get paged in, wasting ram.

[jake]

Re: Snap is a single compressed file,

These things are built to waste system resources. It is expected, and seemingly celebrated.

[that one in the corner]

Re: Snap is a single compressed file,

> then n copies of libraries get paged in

Where n is the number of those apps that are running simultaneously.

Compared to k, which is the total number of apps actually installed and k > n, by quite a large factor in some cases. Yes, sometimes k = n, particularly in specialised circumstances - but specialised circumstances get specialised treatment (lots of extra RAM if it is a one-off box all the way to hand-built-OS-why-are-we-talking-about-Snap at the billions-sold-and-who-updates-toasters-anyway end of the market).

It is all tradeoffs. I don't care if Blender, LibreOffice, Gimp and Steam etc all have their own copies of all the same libraries 'cos they aren't all going to be running at once.

Anyway, all the loaded libs are vastly overshadowed by the memory needed to open a couple of browser tabs!

[Anonymous Coward]

Re: Snap is a single compressed file,

> I don't care if Blender, LibreOffice, Gimp and Steam etc all have their own copies of all the same libraries 'cos they aren't all going to be running at once.

Maybe not on your computer.

> Anyway, all the loaded libs are vastly overshadowed by the memory needed to open a couple of browser tabs!

Ain't that the truth. If I see massive swapping taking place, my first action is "pkill -9 firefox".

[jake]

Re: Snap is a single compressed file,

"Even embedded SBCs can afford to burn through a few extra gigs if it makes them more reliable and robust."

But do they actually need that many 9s of reliability?

Sure, bespoke systems (spacecraft & etc.) might need this. But your average linux desktop/set-top-box/telephone/SBC? Total overkill.

Shirley at this point we are well past the point of diminishing returns in what is a general purpose operating system.

This 19ish year old HP laptop has never crashed, never lost a byte of data, never been compromised, with no issues updating and installing new software. Slackware 10.0 -stable on ext3 when new, moved to ext4 with slack13.1 in 2010, still on ext4 today running slack15.0 (flirted with reiserfs briefly around slack12.1) ... all without any OS or program related issues.[0][1]

From my perspective, not a single one of these new ideas seems to have a compelling case for inclusion FOR THE VAST MAJORITY OF USE CASES. And yet, here we are. Wasting system resources for no good reason. (Other than the pervasive "it's there, we must fill it!" of the Lost Redmond Generations.)

Unless the companies pushing this kind of thing have ulterior motives, of course. My guess is that it's a control issue... and not the user's control of his own hardware, either.

[0] I have replaced/upgraded the HDD, battery and RAM over the years. She was built just before the second round of bad caps at HP.

[1] I fully expect this old laptop to dissolve into a lump of slag after typing that for all the world to read ... but I have known good backups, multiple fall-over boxen, and the ol' gal doesn't owe me a dime. I'll shed a tear, raise a glass and give her a decent burial when the inevitable happens.

[damiandixon]

Re: Snap is a single compressed file,

I've run into problems on a number of occasions with permissions and defaults while using a snap package.

The normal package worked fine.

[jake]

Re: Snap is a single compressed file,

I've almost never run into a problem with Slackware packages.

The "almost" is because I screwed up when creating a couple of packages early on ... but a quick modification with vi sorted 'em.

[captain veg]

history

After experimenting with Slack and Red Hat I settled first on SuSE as my distro of choice, later moving to Ubuntu and then Mint on the desktop, plain Debian on servers.

After the anarchy of Windows software, just the very idea of package management was a revelation. But I have to say that RPM/YaST would far too often get totally confused over dependencies and give up, leaving the meatware to sort it out. DEB/APT usually just works, for me, given the existence of suitable repos. Sometime the latter is not given. It's occasionally inconvenient, but I now just don't bother with such software. Usually there's a workable alternative.

Snap? Mint doesn't like it; can't say I've missed it. Flatpack? I've not found any compelling reason to use it.

YMMV.

-A.

[Liam Proven]

Re: history

[Author here]

> Snap? Mint doesn't like it; can't say I've missed it. Flatpack? I've not found any compelling reason to use it.

That is one of the good things. You don't _have_ to use any of this if you don't want to.

I talked about `deb-get` when I looked at Zinc, a Snap- and Flatpak-free Ubuntu remix. Experimentally I removed both from one of my own machines, and replaced everything with native APT packages via deb-get. I don't think it saved any disk space, but bootup was quicker, and my `mount` command gave much cleaner output.

It's all optional. Some distros avoid the whole thing. There is still choice.

[Lil Endian]

Oh Snap!

...and my `mount` command gave much cleaner output.

I first encountered snap on a Focal Fossa install. I didn't see it coming (bad I know) but I physically recoiled on my first lsblk, wide-eyed I nearly fell off of my chair! (Slight hyperbole! :D ) I honestly thought I had a major problem with all those loop devices. Snap got ripped out straight away. I appreciate the intention of snap, but there's gotta be a tidier way of accomplishing the objective.

Thanks for the articles Liam - good stuff and much appreciated. Cheers!

[jlturriff]

Re: history

"It's all optional. Some distros avoid the whole thing. There is still choice."

For the moment. Given the responses by OpenSUSE to pushback from the desktop community, very soon it will NOT be optional; and probably not long after that, most of the other distros will fall in line the way they did with systemd.

[Anonymous Coward]

Re: history

It's a justifiable concern. When one of the big commercial backers (Red Hat, Canonical, typically) decides to adopt some new feature or widget into their distribution, it tends to drive it into many, if not most, of the others. At a minimum, the src-rebuild distros follow along basically by necessity/policy, and that accounts for a lot.

systemD was inflicted on the wider Linux communities because poettering sold it to the corporate powers that be (marketing?) at Red Hat. For better or worse (more of the latter in the last decade or so IMO) Red Hat decides what a lot of Linux will look like. Debian might have stemmed the tide somewhat, perhaps only for a while, but it didn't happen that way.

Similar story with Gnome3, NetworkManager, probably Wayland at some point, more to come e.g. snap / flatpak. The tipping point is often when some other dependency or prerequisite around the new widget comes up, and any question of including it or not becomes a moot point.

[Rich 2]

This is why we can’t have nice things

Informative article. My biggest takeaway from it is the state of thinking of Linux software devs.

There was a time when we had relatively simple package installation programs. I can see that there are issues with them so I’m not against improvements.

But the “improvements” are a tangled web of complexity and bloat…

- Having the file system functionality being integral to the installation - ref Snapper

- Having multiple copies of libraries installed - Snap

- Relying on the type of init system in use - Snap

- Having to reboot to install software - ALP. And the idea of running your OS as a VM do that you don’t have to reboot is nuts! Not using ALP would be much simpler and less resource hungry

…etc etc etc…

So the “solutions” to a problem that isn’t really THAT big is to add gloop upon gloop, ridiculous dependencies on stuff that nobody sane person would think necessary or desirable, and huge huge complexity.

It’s exactly this kind of thinking that makes multi gigahertz modern systems run like arthritic slugs and means they need gigabytes of memory just to wake up.

[jake]

Re: This is why we can’t have nice things

"It’s exactly this kind of thinking that makes multi gigahertz modern systems run like arthritic slugs and means they need gigabytes of memory just to wake up."

It's also the reason that after nearly 30 years of Linux, I still run Slackware.

[mpi]

"It's all about making it easier for humans to manage the huge, sprawling complexity of modern apps."

Here is an idea how to do that:

By making apps that keep their sprawly-ness well in check and make best efforts to self-contain their complexity as much as they can, instead of smearing it out over a smorgasboard of libs, file locations, services and scripts.

If apps get so complex, that we need ever smarter package management systems, to hide their complexity behind a smileyface, which goes :-( the moment a servers environment doesn't/can't/isn't-allowed-to conform to that management systems way of doing things...then maybe, just maybe, the problem is more with the apps, rather than the way they are managed.

[jake]

Indeed.

Software that is not housebroken and leaves little bits of itself scattered all over the file system should be avoided at all costs. (Are you listening yet, GNOME, systemd and Wayland? No? I thought not ... )

[Chris Coles]

The real elephant in the room

while everyone here is focused upon the operating system, there is a much larger problem no one seems to have fully recognised; user files damaged by software designed to break into such files to force users into rental of software; alongside the deliberate destruction of user rights once purchased under classic free market rules; that are now no longer accepted. A good example was when Microsoft made the decision to remove support for Paint. What they did was deliberately damage ANY illustration, within ANY previous document created in Word, that had been created by the previously totally law abiding owner of their software; someone who had purchased and was the lawful owner of their version of Word. Recently Microsoft announced that they were going after the last owners of Word . . . to force them into being only able to access their previously totally lawfully created files, by renting access to 365.

A very good simile would be the historical ownership of fine china, where to force the previous purchasers of fine china to purchase new items, the manufacturers climbed through every window in every home and smashed their previously purchased fine china. For that is EXACTLY what the likes of Microsoft, and others, have been and continue to do. Of even greater importance, many others now do the same thing, damage the original document to force the use of their new business model upon users.

We need software that will protect the original documents, to remove all the externally applied damage; so that the owner of their work always has full access to it for the future.

[jake]

Re: The real elephant in the room

Redmond and other commercial/proprietary software houses are orthogonal to this conversation.

Note that I agree with you (have a beer!), but it deserves it's own conversation.

[Chris Coles]

Re: The real elephant in the room

Agreed, except that that will require The Register to set such into motion.

[jlturriff]

Re: The real elephant in the room (slightly off-topic)

"...totally law abiding owner of their software;"

Never forget that you almost never OWN software of any type that you acquire from anyone (including, ironically, Gnu/Linux), you only hold a (revokable) license to use it. Yes, the vast majority of open source providers will never jerk the rug from under you, but there have been instances where open source software has been unexpectedly transferred to entities that were not so friendly. And remember, we're talking about other things than executables, e.g. ebooks: If an ebook provider (Microsoft comes to mind) decides to terminate their ebook library program, poof! the books for which you purchased licenses are just gone (unless you've hacked their library to obtain a private copy, thereby technically becoming a felon). You otherwise have no recourse, because the library no longer exists.

[Anonymous Coward]

Ah....a response from the author!!

@Liam_Proven

So.....a response to (maybe irrelevant) comment about Puppet......some of the other comments might be considered somewhat more substantial.

Just saying!!

May be AC..........but paying attention!!!

[david 12]

journaling file systems

First, some proprietary ones were hacked out of commercial Unixes: we reported on SGI's XFS in 1999, and IBM's JFS in 2000.

Can you spot the elephant in the room?

[Wzrd1]

Bullshit is a problem

With so many silos, one gets security issues and those eventually spread like a fireworks display across the field of view and beyond.

Every time open source went with silos, commercial style, security took a back seat, just as Ubuntu released a new LTS version without it's FIPS and security enhancement package.

Welcome to the new world, the same old one, just with a new fake name for the two dollar whore.

Maybe next, we'll get Hitler 2.0 or something equally idiotic, to rinse and repeat, with much larger losses, as has been the trend for idiots who refuse to learn from history.

Of course, the correspondent also failed with a certain former German brand of Linux, bought and fouled hopelessly up by Novell, then returned to spender. At least they did touch on the ReisderFS debacle and the tossed baby with the bathwater, when there was no other real game in town.

Yeah, we old farts are really inconvenient and stand in the way of regress, erm, "progress".

Welcome all to the newest, most improved silo, all of which previously failed, but this will succeed like a fine Tesla autopilot or autodrive...

It's a shame we outlawed euthanasia...

I'd sign a lot of idiots in business up for it.

[Someone Else]

Re: Bullshit is a problem

Maybe next, we'll get Hitler 2.0 or something equally idiotic, to rinse and repeat, with much larger losses, as has been the trend for idiots who refuse to learn from history.

At the risk of a (probably inevitable) Godwin backhand, methinks we already have him...at least here on the left side of the pond. And he's running again!

[Claptrap314]

"Will it run Quake?"

Whenever there is some new CPU architecture that comes out, there is always some commentard that makes this point--it doesn't matter how cool something is, or how powerful. What matters is that users are satisfied.

NONE of these tools, after apt-get or yum, are helping me in any noticeable fashion. Not Chef, who thinks that it is possible to take "self healing" from biology and apply it to the world of 1s & 0s, and to do so without having absolute control over the system. Not Terraform, that thinks it's a sin to test files written in a Turing complete language. Not K8s, that wants to take Google's solution to Google's problem and make it my problem. And certainly not snap, that introduces itself by violating 50-year old conventions, or systemd, whose premise is that I'm stupid.

I need to be able to take my programmers' code and turn it into an app in an image that I can test, then deploy to my fleet while maintaining sufficient visibility to know about and fix issues before my customers have any clue that something happened. That's it. And, I'm lazy. I want to be able to create and maintain my one-liners to do this without a huge amount of work on my part.

What I DO NOT need is some Microsoft employee's vision of the world being forced on me. Or someone's attempt to violate the most basic rules of system management being forced down my throat. Snap & Flatpack don't look or smell like tools that want to solve problems for me. They look like tools for someone to snatch control of my system from me, while giving me very little in return.

[Anonymous Coward]

Re: "Will it run Quake?"

> or systemd, whose premise is that I'm stupid.

Oh come now, surely you know that simply isn't true.

systemD isn't even aware of your existence in the first place, and wouldn't care at all if it were.

It reflects its creator in that regard.

[Anonymous Coward]

Re: "Will it run Quake?"

> Snap & Flatpack ... look like tools for someone to snatch control of my system from me, while giving me very little in return.

That's not so, Snap gives you lots and lots of loopback mounts.

[EricB123]

Eureaka!

"For any readers who aren't speakers of British English, the tool's name, git, is UK slang for an annoying or uncooperative person."

So THAT's what "git"means! As I have never been to Britain, I had no idea. Although I do know you guys call math as "maths".

[John H Woods]

Re: Eureaka!

Yeah but you folks call sport "sports" so we're even.

[Anonymous Coward]

Re: Eureaka!

You might also hear "Git!" used as an exclamation, as a truncated form of "Get out of it ya bastard!".

[Anonymous Coward]

Linux, what a house of cards.

[vermaden]

ZFS Boot Environments

If Linux would like to bulletproof then they would implement ZFS Boot Environments that FreeBSD has with beadm(8) or bectl(8) tools.

With ZFS Boot Environments you can change any aspect of the system/upgrade everything/change packages and even leave system in totally broken state ... and just reboot to one of other Boot Environments that you created earlier - once you 'got it' how awesome and useful that is - you will want that everywhere.

Some more info here: https://is.gd/BECTL

Regards.