Linus Torvalds pines for header file fix but releases Linux 5.8 anyway Linus Torvalds has released a new version of the Linux kernel. The meta-maintainer last week pondered an eighth release candidate for Linux 5.8, but on Sunday decided “it's not just worth waiting another week when there aren't any big looming worries around.” Torvalds did encounter “annoying noise with header file …
I love programming in C (99) but I have to say that one of the things I'd throw out is everything that leads to header file dependency problems.
If I #include a file... I've included it. If it was already included, then I don't need to include it again. But, no, you have to put magic #ifdef's through 50 files to stop that happening.
If I #include a file, and it has a structure or other definition in it... then that structure of definition is defined. I shouldn't have to play games so that it is prototyped in files to let them know it's defined (even though they think it isn't yet) so that when it is actually defined, they know where it is (and god-forbid you define it twice).
Then the real magic comes when you want to include structures inside structures, when you want nothing more than a pointer to a structure inside a structure that you haven't defined yet (you don't need to know what type it is yet... it's just a pointer!), so you end up playing games with void pointers that then get cast to the proper type.
Then you have things like you miss off the include in one file - while processing 50 files simultaneously. There's no "I think you meant this, and I can't see any conflicting definitions" logic. Then you have "Do I define this structure before I include that other file, because it needs it, or do I do it after? Do I mess up my nice list of top-of-file includes just to resolve a header dependency issue?"
I actually kept a screenshot I got while coding a game with a copy of GCC - the whole code compiled except for ONE error. It said precisely this:
expected 'struct Player *' but argument is to type 'struct Player *'
Now, it doesn't matter how many times you read that, it's nonsense. It wasn't a typo, or similarly named but different structures, it wasn't a formatting error - the "struct Player *" in that error line is 100%, absolutely, totally identical (I know, I compared them in a text editor!) and there's only a single definition of that structure in all the code. It came about from some weird header dependency where it thought it had a perfectly defined struct Player, but had somehow managed to include the file twice, without it realising that it was the same file/structure inside. So instead of warning about duplicate definitions, the original "player.c/h" files believed they had the definition, but the function they were used in somehow included them via a roundabout route that thought that *it's* inclusion was the correct definition. And the compiler had obviously given the same structure, only ever defined in only one file, two different "internal" names that it thought were different.
It's 2020. I understand the *legacy* of things like C header file inclusion, the preprocessor, etc. but I do still think it's about time that we got a compiler switch which basically means "just sort it out". If you don't have a definition, hold onto that thought and come back to it later in the compile. If you have double definitions but they're from the same file or the structure is identical, maybe warn but carry on regardless. If you need the size of a structure pointer - it's a pointer. Put it in as a pointer. Worry about the structure inside it later. If I #include but the file is in a slightly different sub folder (e.g. #include "SDL/SDL.h"), work it out... warn me or let me choose as necessary. "Did you mean SDL2/SDL.h? Would you like me to modify the include line to reflect this in the future?".
There's no reason that a C compiler, even in "C99 mode", can't take the whole compile as a process, and colourise parts as it goes, leaving unknowns in a grey area until it can determine what was actually meant. Then filling in the gaps later piecemeal (not on a file-by-file basis as that would just leave you in the same position because it can't resolve the entire file, so nothing else is "ready" yet either). Then warning/asking the user exactly what they want to do.
Especially with things like preprocessor file paths (is it SDL.h or SDL/SDL.h?), it could just search, and tell you, because those things might well change from every different computer that code is compiled on. Work it out, present options, because those options are only ever going to affect me anyway, if I'm the one with the weird-arse include layout.
The problem with C is that it's a fabulous language to program in, but far too much time is spent faffing getting compilers, linkers, Makefiles (YURK! Why are we still using that junk?), etc. to work. I use it on several platforms, with several different architecture targets on each, with all kinds of libraries, and in several ways (command line, Makefile, CMake, Eclipse IDE, etc.) and it's the damn setup that takes most of the time to make sure that the libraries are linked when you use their headers, that they link in the right order to fulfill the stupid compiler's demands, that header inclusion takes place in the right way, etc.
It should just be... compile main.c - it includes a bunch of files, we know they are there in the same folder, pick them up and use them if we've obviously intended to, inform the user of what you did, link it with the libraries that included those headers (it's really not hard to have a #pragma or similar for this... if you include SDL.h, link against libSDL, if you include SDL_Mixer.h, link against libSDLmixer... it would take one line in the header!).
I get the reasoning, I know there's the chance of including the wrong version header, library or whatever, but that's not unresolvable (especially if the compiler/linker tell you what did!), and it's very easily avoided.
But recursive-header-includes are a pain in the butt. In some projects where I know the code isn't for public consumption, I just have a master header file and include it on every C file, and that master file includes everything else "in the right order". Why not? Literally a hundred files with just "include "main.h" " and then let the compiler sort it out. But that's not what the header files are designed to do... I should only be including what I need, I know. And the compiler shouldn't be as dumb as a bag of rocks when I miss something. But if it's going to be, then I'm going to make IT do all the hard, unnecessary work, because I'll be damned if the tool I'm using to make it easy to write tools I'm going to use is just going to spew errors at me and expect me to do the manual legwork. It's a machine... it should be doing that for me.
There's a reason most of us wrap header files in multiple inclusion protection #ifdef/#define wrappers. Current compilers are also smart enough to automatically avoid multiple inclusion given the right flag.
Doesn't fix circular dependency loops though.
You've not experienced the true horror of dependency hell till you've used C++ templates ;)
I always find it better to think of C as one giant file, as that's essentially what it is after preprocessing, can sometimes help you visualise how it all goes together.
Also, IIRC in C you can declare an empty struct, use it as a pointer member, then flesh it out later, something like:
struct foo;
struct bar { struct foo *fooptr; }
struct foo { int someval; }
However when I just tried this clang was actually clever enough not to choke on the forward ref, so it may just be that being 'clever'and YMMV. (Can't be arsed to fish out a GCC to investigate further).
FWIW, while C can be funny like this (and to be fair it is getting on a bit as a language; I mean an E type is lovely, but you have to accept the wipers don't work well, and you'll be cold in winter), it's not just a C problem, dependency resolution is a right pain. (increasingly so in todays 'Don't write it, assemble it.' programming culture).
I was a Java dev for a few years, it has some lovely build tools, but I've still had the inevitable A includes B, C includes a different version of B and I really need A and C, but the Bs don't like each other.
Also, the 'Why can't the compiler just do what I intend' wish is something we've all had, but as you point out, it's non determenistic, and that's something you really don't want, as it makes things a right pain to debug. (Trust me, been bitten by 'clever' tooling more than once over my career).
You're spot on with Makefiles too, they're just obtuse... however, they are just one of the tools available. It'd be quite odd but you probably could use something more modern. if you wanted. (Personally I've just tried CMake for my latest, that's quite cool). Also used Qt's qmake in the past but I think that's possibly a little crufty for non Qt or bigger projects. There's also things like Ninja. (Which can be targeted by CMake).
... In some projects where I know the code isn't for public consumption, I just have a master header file and include it on every C file, and that master file includes everything else "in the right order". Why not?
Why not indeed. Sounds a bit like 'Windows.h' to me. Perfectly valid technique.
Gradle always amuses me. Change 5 lines of Java code, download massive amounts of Gradle updates to build the 2Mb app!
I have wondered over the years whether the build systems people seem to come up with these days should really be several times larger than the OS they run on.They also seem too separate from the compilers and seem to try and reinvent things that perhaps asking the compiler what it thinks would not be a better move.
As for headers being re-entrant its normally a sign you need re-engineer some earlier stuff that you didnt realise would be a problem later and all it takes is an army of Tardises to sort out.
IBM also did that back in the '60's. It was called PL/I (or PL/1), and when ever you screwed something up, it would oh-so-helpfully "fix" it for you, generally by making things worse than the original error, and often-as-not hiding the "fix" it made.
Notice that no one is programming in PL/1 anymore?
I buy a computer to do the hard, repetitive, boring work for me.
If it can't build a tree of the files, their dependencies, duplicates, and then work out which bits it needs from where - even via brute-force, then what's the point of the compiler/linker, really? And most of these problems are not circular dependencies - even when they are, they are solved by taking files partially rather than in a chunk. If B has a dependency on A is in file A which includes B at some point - that's easily resolved. You have everything you need.
A proper circular dependency is literally un-compilable by anyone, ever. These file-level dependencies aren't - proven by the fact that you can eliminate them by excluding one of the files which isn't strictly necessary, or juggling the include orders. Parse file B. That needs parts out of file A. Parse file A. When you get to the bit that needs a definition from file B - oh look... it's just below the bit we're working on... if we hang on a moment, or look ahead, we'll get the answer. Or even just mark that as "grey", carry on parsing the rest of the file, then parse file A in its completeness and when it comes back... oh, look, we only needed this part out of B anyway and everything else is "not grey" now. Now resolve.
Some compilers can do it. Some systems have had this for decades. Most software still gets written in C. GCC and associated tools are the most-popular/complete C compiler / linker / etc. Why is this not a solved problem?
Is it really that difficult? Scan file A top to bottom, pull out all the named symbols and dependencies they have. Do the same for B, C, D, E. Now that you just have names and dependencies, resolve all the ones you can, whittle out the rest like you would a logic puzzle in a magazine. 1st stage you'll eliminate from consideration 99% of the entire codebase. Then it's just a handful of dependencies, and they'll mostly work themselves out via iteration. Anything left - ASK THE USER. It'll be a duplicate definition (i.e. same thing specified in two different files), a true circular dependency (struct A includes struct B which include struct A, which can never, ever compile anyway), or a false circular dependency (struct B includes a pointer to struct A, which contains a struct B, but it's just a pointer so who cares?).
Literally, you could do this on perfectly compliant C code just by building a list of all the includes in all files, ignoring where they were *included from*, finding them yourself and parsing them just like files A, B and C. And then if you still spot an undefined symbol... have a quick shufty though a centralised database of all the include files in the standard include folders and see if you can make a suggestion. The database for a whole system with SDKs etc. and thousands of include files would be, what, a couple of meg or so (just the file name and the symbols it has defined in it).
Well, abuse of macros.
There is a "fake header-only" programming style where you have to include the header (at least) twice, one of which has to be with a special macro defined, and the rest must not. Popularised by the STB libraries I think.
It's now mostly used by people who think they're very smart but are actually very foolish, as it's basically impossible for IDEs to highlight and the worst possible case for the toolchain.
A compiler that tried to work out this ridiculous mess would never be able to compile it at all.
I don't understand what is so wrong with Makefiles? They are great when you want to build on a machine that isn't yours. For example, I code on my PC, but test on a sever (or other people compile onsite) Makefiles are great for dependency checking.
For example, My Makefiles let the user know each step of the process with short descriptive messages, combine the source from the src folder and the includes from that folder (and sub folders) it also only re compiles only the needed source files if I make a header change.
Admittedly there is some dark magic in the regex to make some of that happen but once written, it's done. I generally just reuse the Makefile with minor modifications on new projects.
Have they committed anything to gcc or llvm recently?
If Google do drop Linux for Fuchsia, well that'd be billions of mobile devices moving away from using Linux.
I can't see Linux falling out of favour in server applications though. Whatever issues one might have with Linux's license and / or design and how it pans out on mobile devices, those issues don't impact on servers one little bit.
And with Linux seemingly being open to using Rust - presumably possibly encompassing a wholesale move over to it, eventually - there's a chance that the Linux world will start to incorporate Rust's benefits in their security strategy. It remains to be seen whether or not that happens of course. But it seems that multiple OS teams are beginning to think about Rust at about the same time, Linux included, so I think there's a good chance that Linux won't get left behind as a pure-C dinosaur if Rust prooves to be the way to go. So it will probably remain relevant for a good while yet.
"won't Linux become an irrelevance?"
Just because Windows first adds a Linux subsystem and then becomes Linux does not mean that Linux ceased to exist.
But it's nice to see the forward thinking marketers are preparing for the inevitable Microsoft Linux desktop revolution.
There's a reason even the Linux dictator-for-life has been mulling over Rust. David Braben's criticism of systems-level programming languages is a fascinating read. C++ versus C is a never-ending argument; though I would tend to suggest that the 'unpredictability' of C++ when mis-used is an issue for OS programming in said language. C clearly has no shortage of bad code submissions too; and the advantages of the dictator-for-life in control of what does or does not make the Kernel is perhaps symptomatic of the language and compiler's ability to deal with crud.
One does somewhat feel that despite Linux' established base; that there is probably space for a research / hobbyist OS to be built from ground up using modern paradigms.That is, after all, precisely how Linus got started with Linux; out of desire to use the x86 chipset in full and expand a bit beyond the capabilities of other educational OS of the time (Minix being prime example). At the time, C happened to be the best compromise available between programmability and performance.
You might like to take a look at Redox OS - written in Rust.
What's impressive about that project is how little time there was between inception and having something semi-decent running; really, not long at all.
Then there's the option of re-writing Linux. Or Windows. With C OSes I think you could have a rolling programme of conversion of the code base from C to Rust - AFAIK the two languages integrate just fine - and eventually end up with a predominantly Rust code base implementing essentially the same OS.
None of that means that the Userland also has to be translated - sensible though that might be here and there - but with a Rust kernel one could be confident that the kernel's memory correctness was more assured.
You might like to take a look at Redox OS - written in Rust.
What's impressive about that project is how little time there was between inception and having something semi-decent running; really, not long at all.
Not really ... writing an OS isn't that hard in any language, and Rust is easily complete enough to crank out a basic OS. It's persuading people to write (or port) applications to run on a new OS that's really hard.
Nobody wants to use an OS that has no applications, and nobody will write applications for an OS with no users (unless, maybe, it's the only OS that runs on some new hardware that is really shiny).
Redox OS is an interesting system, but its claim to have a "full set of applications" really means no more than it has a basic set of OS tools -- we're a LONG way from seeing it compete with Windows, say. It's good to see that it's been done, and a good demonstration of the capability of Rust, but really not that remarkable.
"that there is probably space for a research / hobbyist OS to be built from ground up using modern paradigms"
I see theres a microkernel called redox os written in rust... its early days... theres no usb drivers yet and form what limited experiance i have could only emulate it ....
I also read that Linus Torvards said microkernels were a bit pointless...and that he could see kernel drivers being written in rust
https://en.wikipedia.org/wiki/Pragma_once solves it for me :-)
Yes, but only on compilers that support it.
Modern compilers that do support it will generally handle conventional include guards just as efficiently, and on old compliers that don't support it -- there still are a few -- you can't use it anyway.
As that Wikipedia link points out, there are some caveats concerning #pragma once .
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
