Posts by containerizer 71 publicly visible posts • joined 23 Jun 2023
> That dumb lawsuit against Google notwithstanding
There was also the thing were they went around aggressively auditing everyone for use of the Oracle JDK and sending them a huge bill. From then on, most of us have considered their JDK build to be radioactive (despite recent license changes promising that it isn't).
Preach brother/sister.
These days I am mostly in the IDE, and I use it for visual reviews and commits, but pretty often I use the same commands via muscle memory that I did when I started with git 20-odd years ago. add, commit, push/pull/fetch, merge, rebase, log, diff. It helps to understand what the commands are doing to its 'datastore' underneath, and how it copes with blobs etc.
A good habit is to make good use of branches, regularly push 'em to the server, and delete them when you're comfortable the work is merged. That kind of workflow is second nature to me at this point.
I found subversion less intuitive to use when I was forced to use it on a project (git-svn is a real lifesaver). Complex merges just made it fall apart, and it was incredibly slow. Once you are used to git's speed and immediacy - pretty much everything is instantaneous - it's hard to go back.
> git is a sequential forward change management, so Linus can forget all the previous rubbish. until you need to cherry pick and backport changes to an ancient ABI broken kernel for an embedded design, then you live in backport hell
But that isn't a git problem. It's a "managing legacy code" problem. No revision control tool can do this for you. You have to manage your codebase and processes very well, most people don't bother.
> Git reminds me of Cisco's switch OS: cryptic notation and many handy shortcuts for the geniuses who wrote it.
I am starting to wonder about myself. I am not a genius, and I make stupid mistakes all the time, both when using a computer and when not. Cisco IOS feels intuitive and obvious to me, after I spent a few weeks playing with it.
When someone says "genius" to me I think of someone like Linus Torvalds and his contemporaries, or devs like Fabrice Bellard, people who manage large and complex codebases and turn out crazily brilliant software that most of the rest of us couldn't begin to comprehend. That's what a "genius" is. I couldn't come anywhere close to this kind of capability.
I think it's more likely that I am not a genius, and IOS isn't that hard. I wonder are we breeding a generation of engineers who think that if they cannot figure out how something works in ten seconds then it's too hard.
> Navigating Linux, FreeBSD, and worst of all Git is hard, especially the last one.
I'd like to think this is a consideration that does not apply to someone who has commit privileges to the FreeBSD project, a thing that they don't just hand out to randomers.
Isn't it more simple to conclude that a human made a mistake ? And yes tools should try to make mistakes unlikely but accidents happen. Whenever there's a car crash we don't start a public debate over whether or not cars are too hard to use and if we shouldn't just use bicycles instead[1]
> Personally, we always found Git much harder work than the tar command.
Personally, I found Lord of the Rings much harder work than The Very Hungry Caterpillar. Some people suggested that LOTR might have more stuff in it, but screw them.
Anyway, haven't you ever accidentally included a file you didn't expect when you created a tar archive ?
> It should come as no surprise that multiple teams are working on compatible alternatives.
All of these efforts are ultimately doomed to failure due to the reality that most other people have discovered, which is that it's a hell of a lot simpler to simply learn git. Yes, it's esoteric and weird, and conceptually hard to wrap your head around. But millions of devs have done so successfully. There's an entire generation now who do not have to deal with stupid things like Clearcase, CVS or any of the other myriad tools - in many cases far more expensive and complex than git - that popped up over the decades solve around this problem. Nowadays there is only one tool that nearly everyone uses, and all you need to do is invest some time and learn it. It's fast, works consistently and is available everywhere, and there's some wonderful tooling wrapped around it that makes stuff that used to be really hard into a breeze.
I'm not the world's cleverest person or the world's best developer (by a country mile) and I managed to figure it out - it took a bit of effort and practice, but most worthwhile things do.
[1] well, some people do. But nobody listens.
> This distinction makes FreeBSD’s project governance more resilient to individual changes.
Not trying to troll, but it's probably also why FreeBSD hasn't captured mindshare in the way Linux has. Linux did have a head start, of course, but people didn't feel strongly enough about the governance model to walk away from it.
It's fair to say that the benevolent-dictator-for-life model falls when the benevolent dictator is an idiot. That happened with GCC/EGCS a long time ago, and I'm sure in other cases too.
> I also remember the ongoing saga of Microsoft trying very hard to basically co-opt Java into a Windows only thing, and they did kinda succeed in the sense that there really was Windows-only Java code. But then our pal Larry sued, so that was that. I still didn't really care either way.
This happened in the late 90s/early 2000s IIRC, a few years before Oracle bought Sun.
> You have just given away as someone who juses Java because told so, and because Sun relesease it for free. And knows nothing about it really. That is the only reason why Java became so popular.
Nonsense. Free C/C++ compilers, and other useful languages like Perl, were widely available at the time when Java began to gain traction in the early 2000s.
> The Java community's general insistence to resort to XML based dependency injection to circumvent the type system's diktats are a motivation here. But that's far from a desirable feature.
FWIW that's about 20 years out of date. The DI frameworks (of which the widest used is Spring) handle this with annotations, much neater and easier to maintain than XML.
> From what I read, Rust is a far cry from a Java-like garbage collector
Rust is a programming language, not a memory allocation technique.
Nothing stops someone from implementing a garbage collector in Rust, or for that matter in C or C++. Horses for courses.
> which essentially allows memory to leak until you run out of memory then makes everything wait while it tidies up
That's not fair. Modern GC implementations can be used which do a lot of clever stuff in the background. They can't reduce the delay to zero, but you can tune high-volume applications so that the delays are measured within single-digit milliseconds.
It's a skillset in itself to understand GC and learn how to measure and tune its behaviour, so I won't downplay the fact that it's overhead. But there's a different skillset needed for managing memory in C. Assuming that you don't have bugs that leak memory, you still need to consider things like memory fragmentation. The memory allocation routines are not instantaneous - they do have to spend CPU cycles looking for blocks of memory in the heap that match the size you requested. Linux/etc allow you to avoid this by allowing essentially infinitely large heaps and falling back on the virtual memory subsystem to deal with it, which works well but is wasteful.
> On the other hand, Rust fans seem to sweep bounds checking (that is also an aspect of Rust) under the rug or insist that everyone else does it and you should just get with the program.
I've no idea what this means, but Rust enforces bounds checking at compile time, so I suspect your research is deficient.
> I will say that multiple layers shouldn't exist in a driver that is written for performance. Calling a subroutine imposes a huge performance penalty.
Rust doesn't "call a subroutine" (since when was calling subroutines a problem ?). The memory allocation stuff is done at compile time.
> If Rust was seamlessly able to work with C's ABI, adding Rust code would not be difficult.
Pedantic nitpick : Rust code does seamlessly work with the C ABI.
The problem is the kernel API, or more specifically, the fact that it very deliberately does not have a stable one.
If you try to extend C to make it give the sort of guarantees that Rust has, you would end up with something that looks like Rust and has the same integration issues.
It does.
If someone submits a change that breaks another component, they are expected to include fixes as part of their submission.
If the "other component" they break is something to do with Rust, then they need to know Rust in order to fix it. In other words, kernel developers would be required to master two languages, not just one.
Remember that a lot of devs are volunteers. They're doing this because they enjoy it. Forcing things on them that they don't see the point of is a good way to get them to leave.
The problem with this is that there is no stable "C API" within the kernel. There never has been.
when kernel developers make changes to subsystems that break other things, they are expected to update those other things at the same time.
How does that approach work if the thing they break is the Rust layer ?
The solutions are limited :
- introduce a stable C API in the kernel. Most of the kernel developers won't tolerate that as it adds overhead.
- all kernel developers must learn Rust to avoid breaking the Rust layer. Ditto.
- accept that Rust will be broken from time to time. I don't think the Rust guys would accept this - they'd constantly be playing catchup.
OK, so we've established that you lied about Trump not being invited and now you've pivoted to the idea that the party whose nomination he sought and won was "hostile".
> Look at how Moscow Mitch has done his level best to sabotage Trump.
Yup, refused to lead his party to convict him in the Senate following his impeachment, and cleared the way for his Supreme Court picks. Sabotage takes many forms, it seems.
> The everyday people are sick and tired of being ruled from on high by the billionaire bankster class, the life long politicians who are in their pockets and their cronies.
You're defending a billionaire who appointed several other billionaires to his administration. Is it too much to ask that you are at least consistent ?
> You mean like the debates he wasn't invited to?
No, I mean the debates he could have attended if he signed up to the conditions of the debate. Which he refused to do.
> The people voted in the various primaries and voted for Trump. This is democracy in action. If they wanted someone else they could have voted for someone else.
Thank you for explaining the concepts of democracy and the party nomination process to me, but I was already aware.
> Its not like the RNC just decided that Trump is the candidate and it doesn't matter what the party at large wanted
I didn't say the RNC decided anything. I said that Trump mounted a reverse takeover of the GOP and ended up dominating it to such an extent that he thumbed his nose at their own nomination process and still won.
> I mean, something like that would NEVER happen with the DNC now would it? *cough* Hillary *cough* Harris *cough*. Heaven forbid the party ruling elite just pick who they want!
I think you are confusing me with someone who is defending the Democrats.
As things stand, neither party has this kind of coherent discipline. They don't have the same culture of whips/party line that we do. Heck they had to get George Clooney in to get Biden to step down.
I think it's because the US is so geographically distributed. It also has had, until recently, a culture of slowing the workings of politicians and political institutions. In addition, the party's elected representatives have limited control over who gets nominated for the presidency, which is why Trump (historically a registered Democrat) was able to mount what was effectively a reverse takeover of the Republican Party. His takeover was so complete that he didn't even bother going through the motions of campaigning to be the presidential nominee.
If there is any justice, what follows will be a rout of the Republican Party which will lock them out of power for several decades. This happened in the 1930s following the Great Depression.
I appreciate the lesson in structured computer organisation, but I am still struggling with whatever your point is.
If you have two languages, one is safe, one is less safe, and you run them both on the same hardware, isn't the safe one less likely to have bugs ?
(FWIW I've been programming embedded systems for > 20 years; I've written the odd value to a memory location in my time .. )
> You have to develop your own safety for an operating system,
No, you don't. That's the whole point.
> C is perfectly safe as long as you know what you are doing.
This is true of a lot of things considered dangerous for humans.
> but there is no way to do all the things that have to be done and still have a compiler run the safety
Excepting memory-mapped hardware, have you got any examples ?
> Safety has to be hardware dependent.
No it doesn't. There are many widely used memory-safe programming languages.
They are static, but they are also things which are only necessary in a type-unsafe language. You don't need address checking in a language that validates address accesses at compile time. You don't need undefined behaviour checking in a language that has no undefined behaviour.
The second sentence is effectively suggesting that Rust's capabilities are not proven to work. Given the widespread adoption and the fact that Linus Torvalds thinks it is worth at least trying it in the kernel, I think the burden of proof is upon you to show that all these very clever people are wrong about Rust, not anyone else.
>The whole reason people are using rust in the kernel is because it makes also makes some kernel code easier to maintain.
It makes some kernel code easier to maintain for some people, and makes life harder for others. It means that instead of having to learn one language, kernel devs have to learn two, because now any breaking changes they make could effect the Rust layer as well as the C codebase.
> it's an announcement that he plans to sabotage someone else's work, not because it has a technical flaw, but because he doesn't like it.
This is a mischaracterisation of what Hellwig said.
He didn't say he didn't like Rust. He said that maintaining two codebases in the same kernel would be too difficult and would threaten the future of the kernel.
I don't think he should be disciplined for saying this.
> THAT is exactly the point I was making - it is the application that must make this decision. Not the kernel.
I don't understand. What decision ?
select/poll/epoll do not make any decision. They simply cause the application to block until certain conditions (eg arrival of data) arise. Whether or not that condition is detected via the arrival of an interrupt or via polling is hidden from the application.
Slight nitpick on an excellent comment : enterprise NICs (and most of the good cheap ones) do DMA, so there is no issue with getting packets out of the NICs internal buffers. This is usually set up as a ring buffer and you can usually configure the size. Of course, you absolutely can get issues if you fail to service the ring buffer quickly enough.
Avoiding dropped packets then becomes a case of tuning the ring buffer size and the parameters which flip between interrupt and polling mode.
No, you definitely do not want PREEMPT_RT in most enterprise settings where maximising workload per watt are prioritised. PREEMPT_RT significantly reduces throughput in order to improve latency. This is rarely what people want.
Applications indicate they want to read more data by notifying the operating system via select()/poll()/epoll(). These mean "stop me, then wake me up when more data is available". The details of how to discover whether or not data is available are hidden by the kernel so the application doesn't need to care, it just needs to correctly use the proper API. Note that the poll(), epoll() calls are not a direction to the kernel to use polling instead of interrupts.
Imagine it's a web browser waiting for a response from a server. Simplified, the application opens a socket and sends data to the server and waits for the response by calling poll() on the socket.
When that happens, the kernel kicks in. If there is no data on the socket, the kernel won't reschedule the application. Later, the server responds - an interrupt occurs, the kernel reads the data and then places it in the application's socket buffer. The application is then rescheduled and can immediately read the data as soon as it is resumed.
In the alternative world with polling, the application does not change. Instead of waiting for an interrupt, a timer inside the kernel periodically wakes up and checks for data. If it is there, it follows the same series of steps that it did when it got an interrupt. The application does not have to change.
Some network cards, typically the higher-end ones, already effectively support this with a feature called "interrupt coalescing" where they will wait for their buffers to fill up, or for a timer to expire, before notifying the CPU.
The approach mentioned in the article is likely to be beneficial in high throughput scenarios, but not all. There is a crossover point; if your I/O is frequent and regular, polling is more efficient than interrupts due to the extra interrupt servicing overhead. If the I/O is more patchy, polling may waste CPU cycles through doing polling work which rarely finds available data, and it may also introduce latency as a packet will have to wait for the next polling interval before being serviced. According to the article they're adopting a hybrid approach to switch between polling and interrupts depending on the conditions, which is clever. Tune that right and I'd say this feature will end up being enabled by default for most deployments.
Interrupts are great if your I/O comes and goes at random, relatively infrequent intervals, which might be the case for compute-bound workloads
the cores may or may not be the same, but it probably does not matter given that they are implemented to comply with an ARM spec.
I believe the other hardware inside the SoC does change around. All the audio, video, network/wifi drivers, graphics etc are on-chip and I imagine they're continuously updated.
> But they are trying to do something no other Unix project has ever done: the aim is that you can stick a HAMMER2 volume on a shared connection and multiple independent OS instances can all mount it at once.
It sounds like you're describing a cluster filesystem. This has been done many times in the UNIX world - Veritas, GFS[2], GlusterFS etc.
> The fact that rust has this "unsafe" directive (or what ever it is called) means that the language designers absolutely know that the language cannot do what people want to do with in in a memory-safe manner.
I can think of very few cases where this kind of operation would be necessary. The one obvious one is where you have to manipulate memory that isn't really memory, such as when you're accessing memory-mapped hardware within a kernel, or certain other low-level operations where something else is handling memory for you.
This does not mean the language is bad. At the very least it shows that unusual memory usage which cannot be tracked by the compiler is at least auditable and the compiler can emit diagnostic warnings when they are used (or enforce against their use). In C there is no standard way to do this.
You cannot build a programming language which is foolproof in all scenarios. You can, however, build one which minimises bugs caused by human error. I think that's arguably a win over a language which pointedly does not.
> The precise issue is that C the language is not "memory unsafe" because it doesn't do memory management, the libraries and apps do that.
I am sorry, but you are very wrong.
Managing the stack is a form of memory management and the C language does this. If you declare a variable or an array etc on the stack, or take a pointer to it or dereference it, the compiler generates code for this purpose. Off-by-one errors and other stack-related bugs are a very common cause of instability or security bugs. That's a fundamental feature of C.
The use of malloc() and free() are discussed in the original K&R book and are defined in the ISO C99 spec (safe to assume they are in ANSI C89 and in subsequent ISO specs). They may not be operations that are directly converted into machine code by the compiler but that doesn't mean you can get away with saying they are not part of the language.
If you want to talk about losing credibility, scoring pedantic points by trying to suggest that the language and the support library which forms part of the language specification are not intrinsically linked seems like a good way to do that.
It would take several pages to explain in detail all the different ways where you are utterly wrong. But in summary :
- I really do not know where to start with the notion that only CISC CPUs can perform "complex mathematical and algebraic operations" with "extreme precision and efficiency". SPARC, MIPS and PowerPC have lengthy track record here, being used for CGI in films, by the oil & gas industry, financial services sector etc etc.
- A RISC is not inherently a "low power part for small form factor devices". The earliest RISC CPUs were used to build servers, workstations and mainframes. IBM dominated enterprise computing with the RS/6000 workstation, and its S/390 CPUs were a CISC ISA running on a version of its POWER ISA RISC platform.
- CISC does not mean "able to do complicated things". CISC means "I have a complicated instruction set whose instructions may take several clock cycles to execute and which you may never use".
- I have no idea why you think the inability to emulate another instruction set at full speed rules out an architecture as being viable.
- the Motorola 68K and Itanium are not RISC architectures. 68K is "dead" because it can't run Windows, and Itanium was simply a poor design.
I remember life 25-30 years ago. Nobody in their right mind would have deployed x86 in the enterprise server space, it simply was not done. Every RISC CPU wiped the floor with x86 at the time. They lost because x86 was cheaper and could run Windows, and Intel were eventually able to hotrod their rubbish architecture to make it run fast.
These days, the CISC vs RISC thing does not matter. It was important in the 1980s/90s when chip real estate was at a premium, and RISC could use the space vacated by complex instructions to make simple instructions run much faster. Nowadays, everything including x86 is implemented on a RISC core with the higher level CISC instructions microcoded.
I don't recognise the idea that memory safety only recently became important. People have been banging on about C's limitations in this regard since the language first became available.
It's not like the industry has been frozen in aspic. Outside of specialist fields (kernels/device drivers, low-latency software) C/C++ have been replaced with Java, C-Sharp and Python, and Go is making some inroads in the systems programming side.
The remaining software which continues to be in C is that which cannot be easily migrated to any of these languages. For most cases, the cost of continuing to use C is lower than the cost of replacing codebases with Rust. The workarounds for C - static analysis, stack smashing protection tools etc - are deemed "good enough" most of the time.
I can well understand why some kernel developers might see that all of this is a solution in search of a problem. And the Rust crew aren't the first to make this case - there have been attempts to push C++ on the kernel in past years too, and the same kind of reaction when it was blocked - that the devs are a bunch of luddites trying to hold back advancement. It's not that simple.
If the world really is right, and the kernel devs are wrong, then this problem will solve itself in a different way : people will write a from-scratch, Rust-only kernel which is compatible with Linux at the system call layer and can therefore be dropped in to existing distributions. If that idea seems too far-fetched, then it telling us something about the cost/benefit of using Rust.
> In my experience, Rust code is rarely shorter or easier to write than its C counterpart. It takes a lot of discipline to write anything sufficiently worthwhile in it.
This statement is probably not true if the definition of "sufficiently worthwhile" includes a memory safety guarantee.
> Now had the issue of memory safety, etc, in C (or the lack thereof) been addressed by creating a sub-set of C and/or a few features to mitigate some issues like use-after-free then it would have been fairly trivial to do this
"Come on boffins, get of your backsides and make C safe!"
> There is no downside to it other than having to learn Rust, some toolchain issues in some embedded environments, and of course shaking your cane at everyone on your lawn.
But two of of the three mentioned are pretty big issues.
C/C++ compilers have had decades of refinement behind them, work everywhere, and loads of people know how to program in it competently. Even if you managed to persuade people of the technical merits of Rust, the inertia is always going to be there.
With respect, saying "some toolchain issues" is a bit flippant. It looks like only x86 and ARM-64 are well supported ("tier 1"). Many embedded platforms won't have that hardware - I'd expect ARM32 to remain popular for a while yet. Other RISC architectures seem to be withering on the vine - MIPS seems to be dead, although I'd expect there's still a lot of PowerPC in telecoms-focussed SoCs. And SPARC/s390x remain small, but important in key enterprise markets.
A lot of that problem would go away if they'd switch the focus to adding a GCC frontend, where almost every other major language and target architecture is supported.
Don't want to be the "nothing to see here" guy, and layoffs are never good. But some of this actually makes sense.
RH have a long history in maintaining their own builds of things, for example the JDK or Spring, where they add little in the way of extra capability. This made sense back in the days of yonder where community projects tended to be less concerned with LTS builds. RH added value by maintaining such builds.
OSS projects seem, in my perception, to have become much more mature of late, often maintaining their own LTS builds (corporate sponsorship plays its role here). Inevitably, rolling one's own build therefore accomplishes less.
I think it was a couple of years ago that RH announced that the Temurin JDK build would be fully supported on Openshift, for example. There's a win-win thing here; RH backs, and presumably helps fund, a community-led build, so they don't need to have a separate team themselves. The community gets the sponsorship.
I can find no reference, anywhere, for the term x86-32. Did you guys just make it up ?
The 32-bit variant is pretty much universally known as simply x86, or sometimes ia32.
The 64-bit version has been variously branded amd64, intel 64, x86-64 or x64.
There's also a thing called x32, which was an attempt to have a hybrid. It runs on amd64 but limits itself to 32-bit pointers.
Introducing yet more nomenclature is not at all helpful.
You don't even need to do any branching. Just use kernel 6.6, which is going to be supported in LTS form for another 3+ years. Then you can branch it.
But of course branching should not be a problem, as these folks will presumably already have branched their own compilers and distributions all of which dropped support for this arch long before the kernel did ..
