back to article Linux kernel maintainers tear Paragon a new one after firm submits read-write NTFS driver in 27,000 lines of code

Paragon Software is trying to get its NTFS driver into the Linux kernel, but has submitted it as a single dump of 27,000 lines of code, sparking complaints that it is too large to review. NTFS is the default file system for Windows XP and later. Microsoft is beginning to replace it with ReFS for some scenarios, but NTFS …

Page:

  1. KorndogDev
    FAIL

    Why not in 0.5 file?

    Just scan the first 500 lines and count all goto's.

    1. Jon 37

      Re: Why not in 0.5 file?

      Certain uses of goto are a normal and accepted coding practise in the Linux kenel. They use return values and "goto cleanup" to get something like C++ exceptions.

      1. bombastic bob Silver badge
        Devil

        Re: Why not in 0.5 file?

        It also tends to generate more efficient code, if you don't gerrymander things just to avoid using a 'goto'.

        We're talking KERNEL code, here. It's not the same as some GUI "app".

      2. Kevin McMurtrie Silver badge

        Re: Why not in 0.5 file?

        Better yet, use C++ if you think you need a lot of C++ features in C. There's nothing like trying to work on C code that spends 40% of it's time pretending to be C++.

        1. Anonymous Coward
          Anonymous Coward

          Re: Why not in 0.5 file?

          The entire Linux kernel should be rewritten as a set of C++ templates. That'd be cool.

        2. overunder Silver badge

          Re: Why not in 0.5 file?

          "There's nothing like trying to work on C code that spends 40% of it's time pretending to be C++."

          Ha, how about C++ code that pretends to be C++ code? Pointers to member functions seemingly weren't C++ enough, so (this->*that[val])( foo ) becomes something similar to: [ this ] ( val ) -> return best_guess_lamda {}.

          As far as the article, 27k lines isn't that much, but if you present a 27k blob to be maintained.... so, I see it both ways and am on the fence.

      3. arctic_haze

        Re: Why not in 0.5 file?

        Maybe using goto is normal in the kernel but it does not change the fact that it makes reviewing the code an extremely unpleasant task.

  2. Androgynous Cupboard Silver badge

    Bit harsh

    While I get the problem, donating a read-write NTFS module to Linux is a good thing, isn't it? Paragon devs are not kernel devs, but that's hardly their fault. I suspect 20 years ago, someone would have said thanks, jumped on it and cleaned it up for inclusion.

    1. moonpunk

      Re: Bit harsh

      Couldn't agree more - like you say there would have been a time that the community would have been grateful and would have jumped on this to get it included.

      Happy days!!

      1. Anonymous Coward
        Anonymous Coward

        @moonpunk - Re: Bit harsh

        If the community would have been grateful and have jumped at this kind of code back in those days, Linux kernel would have been nowhere today.

    2. anothercynic Silver badge

      Re: Bit harsh

      Have to agree here... Although it may have been useful for Paragon to have engaged with whoever has contributed filesystem drivers to the kernel before and asked "how do we do this best?"

      That's how I got started contributing to open source efforts... Rather start off on a good note as a newbie than having your Nomex pants scorched.

      1. danno44

        Re: Bit harsh

        So Paragon should have their 'newbie' submission accepted because otherwise their feelings might get hurt, and they might not ever share their toys again? Boo-hoo!!!

        I guess when it comes to quality submissions and feelings, feelings win. Let the coddling begin!

        1. sabroni Silver badge

          Re: feelings win.

          The feeling being "I can't be bothered to help merge this, I enjoy moaning about MS stuff too much"?

        2. Anonymous Coward
          Anonymous Coward

          Re: Bit harsh

          There are ways of phrasing things. Instead of just going, "that's too long, lol wtf are you doing you noob", they could instead have said, "thanks for submitting that - it looks like something that'd be really useful. Unfortunately, it's a bit long and doesn't conform to our coding standards, which can be found here [link]. Can we have a chat to see how we can help you get the into a form that can be incorporated into the kernel?"

          1. amanfromMars 1 Silver badge

            Re: How to avoid nearly all of the harsh bits.

            There are ways of phrasing things. Instead of just going, "that's too long, lol wtf are you doing you noob", they could instead have said, "thanks for submitting that - it looks like something that'd be really useful. Unfortunately, it's a bit long and doesn't conform to our coding standards, which can be found here [link]. Can we have a chat to see how we can help you get the into a form that can be incorporated into the kernel?" .... Anonymous Coward

            Quite so, AC, and that is certainly another very good way to proceed whenever into the particular highlighting of peculiar progress and entangling success. Who/What wouldn't find it compellingly commendable and thoroughly recommendable too?

            1. Cynic_999 Silver badge

              Re: How to avoid nearly all of the harsh bits.

              Pleasethankmeformyeffortinwritingthispost. Iwillletyoucleanitupandresubmit.

              1. Borg.King

                Re: How to avoid nearly all of the harsh bits.

                // Pleasethankmeformyeffortinwritingthispost. Iwillletyoucleanitupandresubmit.

                // Thanks.

          2. andro

            Re: Bit harsh

            They did, see this response:

            https://lore.kernel.org/linux-fsdevel/20200815190642.GZ2026@twin.jikos.cz/

            And they have all been working together nicely, the patch and process is much improved, and things are looking good.

    3. Charlie Clark Silver badge

      Re: Bit harsh

      There are quite a few things to consider when you take on the maintainership of the code, which is basically what is happening and, before it gets to a code review you have to be sure that the legal aspects are covered: one of the reasons why there are so few NTFS drivers is because Microsoft has kept the file system essentially private.

      Then there is is the code review. File systems are not new and there might, by now, be a template or at least accepted best practice for their drivers. It's happened before that code dumps were received with "thanks, but no thanks" because there was more work involved in understanding the code than writing it from scratch.

      1. John Brown (no body) Silver badge

        Re: Bit harsh

        "because there was more work involved in understanding the code than writing it from scratch."

        Whilst I mainly agree with you, in this case, no one is going to write it from scratch and the existing NTFS driver has issues. Depending on the licencing terms of this code dump, maybe the devs/maintainers of the current NTFS can just use it top see how it works and "fix" the current implementation, although I suspect not.

    4. Lee D Silver badge

      Re: Bit harsh

      Not really.

      Let me give you an analogy.

      I've got an old banger of a car. It still works. So I leave it in your garden one day with a note saying it's a gift to you.

      The maintenance, updating, API-conversion and upheaval that happens in the Linux codebase is huge. So a dump-and-run is really an obligation, not unlike giving someone a pet dog for Christmas when you don't live with them.

      That maintenance burden is FAR FAR more difficult than the initial code-drop is. You have to understand all the code, change it often, make sure it stays secure and bug-free, deal with support from users who now have NTFS 5.1 and why are they getting corruption now when they didn't on 5.0, and so on.

      Literally, maintaining that code is harder than writing it in the first place. A dump-and-run is a common output from a company that doesn't WANT to maintain it any more, or deal with the user's complaining that it's unmaintained, or has a bug they haven't fixed.

      There were several NTFS projects, historically. Hell, even one that emulated enough of the programming interface to use the original Windows DLLs to access the filesystem under Linux. They've all suffered the same fate - they might "work" but nobody maintains them, so they get bugs and get obsolete and don't work for modern versions of the filesystem, so they end up being a dead-weight in the kernel.

      This isn't unique to NTFS, or even a particular subsystem of Linux - maintenance burden is the determining factor in a lot of the patch acceptance pipeline. Everyone involved has to understand it. It has to use the common code that it can (so NTFS doesn't have it's own special way of allocating a file or whatever that's different to everyone else). It has to work internally in a similar manner. Quirks have to be ironed out so they are clearly documented. Filesystem detection has to be consistent so it only offers to run disks that it knows it has the support for, and tied in. And so the filesystems maintainers have to be able to review it and not accept any NTFS-only nonsense.

      Because Paragon - if history is anything to go by - will not be patching this code in years to come. They'll be dead, gone, forgotten, unsupportive, the one guy who opened the code will move to another company or whatever. That's how *so many* filesystems, drivers, subsystems, etc. die in Linux. Neglect, and nobody able to understand it to take it over, because it's an "oddball".

      All the wifi drivers use the central 80211 framework, which was arrived at after dozens of independent and differing implementations that each vendor tried to put in just to run *their* card, and no other. The commonalities were pulled out, modularised, and then people were expected to conform or explain why they couldn't possibly do that. After decades, all the wifi drivers now use pretty much the same infrastructure even if they are radically different in capabilities, and those manufacturer's are long dead and gone.

      Code-dumps are the problem here. Dump-and-run is seen as a charitable action, but it's just an obligation on Linux kernel maintainers to justify, fix, debug, handle user support, etc. for it for decades. And if they can't understand it, or it does things it shouldn't do, then they can't do that, and they'll reject it.

      20 years ago people did exactly this kind of thing for NTFS. The result after all that time was one central common NTFS driver that's read-only. Because all those other contributors are long-gone and their code was something that "worked" for a brief period but was atrocious to integrate or debug.

      Maintenance is king. Especially in an era where security of something like NTFS, and data security of the user, could easily trash or compromise someone's system and people won't be able to understand how or where it came from. You do *NOT* want a code-dump running your filesystem with any kind of useful data. And you certainly don't want to be responsible for someone else's codedump when your users all blame YOU for their data trashing itself on hardware that you don't even have available to yourself and cannot debug on.

      This isn't a gift. It's certainly not a unique gift, it's happened dozens of times before. It's an obligation.

      1. oiseau Silver badge
        Facepalm

        Re: Bit harsh

        ... isn't a gift.

        ...It's an obligation.

        Great write up. +1

        Summarised long ago by Virgil, ca. 29/19 BC in Aeneid (II, 49)

        Timeo Danaos et dona ferentes

        O.

        1. Paul Herber Silver badge

          Re: Bit harsh

          'Timeo Danaos et dona ferentes'

          Is that the Timeo Danaos who plays for Barca? I didn't know he was with that Dona Ferentes these days!

          1. Sanguma Bronze badge
            Happy

            Re: Bit harsh

            'Timeo Danaos et dona ferentes'

            The timid Danes, fearing even their Donnas! :)

            1. John Brown (no body) Silver badge

              Re: Bit harsh

              I never realised the Danes made such fearsome kebabs!!

        2. Anonymous Coward
          Anonymous Coward

          Re: Bit harsh

          Except.... Linux is falling behind when it comes to file systems. ZFS is now highly desired by many users, yet Linux (through no particular fault of the devs or Sun - it's a consequence of history) has no easy way to accomodate it. Now someone is offering a decent NTFS implementation and the initial response is, "thanks, but no thanks". This "it's too big for us to accomodate" for NTFS response would also apply to ZFS if that were to magically be donated under a GPL license.

          So current criticism of Oracle sounds somewhat hollow; it'd be spurned anyway, even if it was offered with a GPL license.

          The response perhaps ought to be "thank you - hang a mo", followed by a call to arms to raise enough dev effort to properly absorb something this big. Otherwise Linux will forever be rejecting large donations that, really, it ought to be able to accept.

          1. Anonymous Coward
            Anonymous Coward

            Re: Bit harsh

            ZFS is actually quite usable these days - I have a couple boxes running it on root via ZoL.

            1. Anonymous Coward
              Anonymous Coward

              Re: Bit harsh

              Yes I know that ZFS is perfectly usable and very good - it's just not in the kernel mainstream project.

              1. Alan Brown Silver badge

                Re: Bit harsh

                > I know that ZFS is perfectly usable and very good - it's just not in the kernel mainstream project.

                ZFS isn't in kernel mainstream because certain Sun developers specifically DID NOT WANT it to be (apparently there were resignation threats) and the CDDL was deliberately written to ensure llegal incompatiibility

                They work together quite happily, but they can't be distributed together as a working system (dead easy to workaround, just like MS fonts and various other non-GPL bits)

                1. Anonymous Coward
                  Anonymous Coward

                  Re: Bit harsh

                  ZFS isn't in kernel mainstream because certain Sun developers specifically DID NOT WANT it to be (apparently there were resignation threats) and the CDDL was deliberately written to ensure llegal incompatiibility

                  You can't complain about that. It was their code, it was up to them to release it how they wanted. And no, they don't always work together nicely; there's certain people in the Linux kernel community who are keen on making certain symbols unavailable to the ZFS modulues.

          2. BinkyTheHorse
            Stop

            Re: Bit harsh

            This is a megapatch not for "Linux" (i.e. the entire ecosystem), but for the Linux kernel.

            As another commentard already noted below, there are non-kernel solutions for handling NTFS writes which already work quite well, and have been for many years.

            Why should there be a "call to arms" if the prospective contributor has put (charitably assuming) minimal effort into preparing the contribution to be usable, and better alternatives already exist?

            (also, I'm pretty sure there's kernel-level support for ZFS nowadays)

            1. Anonymous Coward
              Anonymous Coward

              Re: Bit harsh

              There is no official kernel level support for ZFS. There is an separate project, from which end users are free to download and install kernel modules to their heart's content, but the Linux kernel doesn't want to, and indeed can't have anything to do with it, due to license incompatibilities.

              Why should there be a "call to arms" if the prospective contributor has put (charitably assuming) minimal effort into preparing the contribution to be usable, and better alternatives already exist?

              Because it's a gift. So far as I can tell the prospective contributor has nothing to lose if Linux declines to incorporate the patch into the kernel mainstream. Why should they bend over backwards to please the Linux kernel community? Ever heard the phrase, "Never look a gift horse in the mouth?". Better stuff for NTFS may exist - for varying definitions of "better", but that's not the point (and any FUSE based solution isn't exactly going to be very performant).

              Linus has already raised concerns about having enough manpower in the Linux kernel project, and it's easy to understand why; everyone needs to make a living, and the munificense of those companies who donate labour can be stretched only so far. If no one is willing to stand up and do extra work, Linux will end up missing out on things.

              1. bombastic bob Silver badge
                Devil

                Re: Bit harsh

                I would rather that they do a FUSE-based solution, and THEN work on the fuse kernel drivers so that they're as efficient as possible.

                FUSE-based solutions would ALSO work on FreeBSD...

                1. Jamie Jones Silver badge

                  Re: Bit harsh

                  I admit I don't use it much, but what's wrong with /usr/ports/sysutils/fusefs-ntfs ?

                  "TFS-3G is a stable, full-featured, read-write NTFS driver for Linux, Android, Mac OS X, FreeBSD, NetBSD, OpenSolaris, QNX, Haiku, and other operating systems. It provides safe handling of the Windows XP, Windows Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7, Windows 8 and Windows 10 NTFS file systems."

                2. Maelstorm Bronze badge

                  Re: Bit harsh

                  FUSE-based solutions would ALSO work on FreeBSD...

                  FUSE already exists for FreeBSD in the ports collection. Besides, Sun Microsystems contributed ZFS to FreeBSD, so FreeBSD natively supports ZFS.

                  1. Microchip

                    Re: Bit harsh

                    Don't think it was contributed, but the CDDL is compatible with the BSD licenses, so people were free to port the code as they wished from OpenSolaris and include it in the BSD distributions.

                    1. Anonymous Coward
                      Anonymous Coward

                      @Microchip - Re: Bit harsh

                      Exactly! CDDL was designed specifically to target Linux. BSD was not a threat for them.

                3. Jaybus

                  Re: Bit harsh

                  Well, there is already ntfs-3g, which is FUSE based. Why should they work on yet another? The problem with that is the huge performance hit, especially with lots of small writes, that is directly related to it using FUSE. A kernel driver for NTFS is not a ridiculous concept.

                  Oh, and btw, ntfs-3g has > 29k lines of code, so the existing FUSE-based FS is actually slightly larger than the proposed kernel driver. For comparison, ext4 has 29k and xfs has 65k. So wtf?

              2. Alan Brown Silver badge

                Re: Bit harsh

                "If no one is willing to stand up and do extra work, Linux will end up missing out on things."

                Indeed. If Paragon were truely committed to this, they'd step up with the manpower to document the hell out of everything and rewrite this to comply with standard formats

                Otherwise it's just a publicity stunt for abandonware

                1. Anonymous Coward
                  Anonymous Coward

                  Re: Bit harsh

                  "Indeed. If Paragon were truely committed to this, they'd step up with the manpower to document the hell out of everything and rewrite this to comply with standard formats"

                  Forget Paragon / NTFS. What would the Linux kernel community do with 27,000 lines of code they didn't like the style of and also required a lot of understanding but represented something they really, really wanted to incorporate into their kernel? Say it was Nvidia offering their driver code base on a take-it-or-leave-it basis?

                  Paragon seem to be copping some bad publicity from this - so how is that supposed to encourage other companies to donate code?

            2. doublelayer Silver badge

              Re: Bit harsh

              "Why should there be a "call to arms" if the prospective contributor has put (charitably assuming) minimal effort into preparing the contribution to be usable, and better alternatives already exist?"

              Do better alternatives exist? Ones that run quickly and support a lot of functionality? Ones that don't require complex installation or configuration for distro devs, one of the main reasons short of performance that stuff gets put into the kernel? I haven't seen that.

              A call to arms doesn't mean that everyone drops what they're doing and starts focusing only on this. It could be much more basic than that. For example, a call to arms among a couple of people who could read some of this code and give advice to the original developers about how to modify it to more easily connect with the kernel and get through a review. Unless the code is worthless, it would seem like there might be some benefits in doing something like that, especially as the original developer claims they're planning to continue to support it. They might be liars, but if they didn't care at all, they wouldn't have donated this in the first place. Maybe it's worth believing them and allowing them to prove it by helping them to make it kernel-worthy. If they don't bother, we're only a few emails down.

              1. Anonymous Coward
                Anonymous Coward

                @doublelayer - Re: Bit harsh

                Haven't you heard of poisoned gifts ? You don't approach the Linux kernel developers team by dumping code at their door and calling the press to tell the world about your charitable act.

          3. Snake Silver badge

            Re: AC, "Thank you - hang a mo"

            I agree. OK, we certainly can understand when the project is initially too large to grab a hold of, but is what Linux devs are saying, in equivalent, "In something large, unless provided as a completely plug-in solution we're not really interested"??

            Heck of a way to grow a community-based development.

            Yes, ideally we would love to see corporate submissions donated with every "I" dotted and "T" crossed. But corporations aren't required, nay really aren't even in the business, of providing the additional manpower to polish a codebase for guaranteed compatibility prior to submitting said codebase free, to a non-profit community-based venture. It's look a bit of the gift horse in the mouth, strictly IMHO.

            Maybe the devs are just communicating their shock (and a bit of horror) of having to review something much larger than their typical codebase. After regrouping they may come about and embrace the submission and be pleased with it.

          4. imdatsolak

            Re: Bit harsh

            But this is exactly what they said. When you read the mailing list, there were efforts such as “let’s break this into multiple steps, so that we can integrate it” OR “please put me on CC when you repost, so I can see how I can help”, etc.

            The kernel devs actually offered help and showed ways how this can be integrated by working together...

            No, I’m not a kernel dev but truth must be said...

          5. danno44

            Re: Bit harsh

            Linux is falling behind in filesystems? Even if that is true, the answer is to accept anything thrown at it's feet, even if what's thrown would crush their feet? I had no idea Linux was so desperate.

            Maybe the devs should stand out on the street with a sign that reads, "Will work for yesterday's filesystem."

            1. amanfromMars 1 Silver badge

              Re: Bit harsh

              Linux is falling behind in filesystems? Even if that is true, the answer is to accept anything thrown at it's feet, even if what's thrown would crush their feet? I had no idea Linux was so desperate.

              Maybe the devs should stand out on the street with a sign that reads, "Will work for yesterday's filesystem." ..... danno44

              That's a tad harsh, danno44. If you can read between the lines of the submission, is it not really seeking out suitable workers for future filesystems with advanced safe and secure systems of remote operation?

            2. Anonymous Coward
              Anonymous Coward

              Re: Bit harsh

              Name the last FS added to the mainstream Linux kernel.

              Effectively it's stuck on ext4, and for some distros its on xfs. Btrfs is becoming a yesterday's fs, and never actually worked properly in the first place. The working filesystems in Linux are dinosaurs in comparison to something like ZFS, there seems little prospect of it catching up (e.g. by fixing BTRFS), because it looks like there aren't enough people who care to put in the hours.

              1. hanshenrik

                Re: Bit harsh

                > Name the last FS added to the mainstream Linux kernel.

                Soon the answer will be "bcachefs" :)

          6. Anonymous Coward
            Anonymous Coward

            Re: Bit harsh

            Yes, it's all about priorities, and speaking of priorities, more importantly is to change terminology from Master - Slave or blacklist, etc..., instead of doing a review what is an excellent driver made by Paragon...

            This Linux community is becoming a true clown world.

        3. this

          Re: Bit harsh

          Tr: Beware of geeks bearing gifts

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021