back to article Following Torvalds' nudge, Paragon's NTFS driver for Linux is on track for kernel

Paragon Software, in response to a nudge from Linux Torvalds, said it will submit a pull request for its NTFS driver for Linux. The process of submitting a read-write NTFS driver for Linux was initiated by Paragon nearly a year ago, when it ran into complaints that its 27,000 line patch was too big to review. Paragon …

  1. Anonymous Coward
    Anonymous Coward

    Would be nice for a little explanation as to what this driver brings to the table that we don't already have.

    Anybody, please?

    1. Boothy

      From reg article Paragon 'optimistic' that its NTFS driver will be accepted into the Linux Kernel

      Quote: "Linux currently has two NTFS drivers, a FUSE (Filesystem in Userspace) driver which is read/write, and a kernel driver which is read-only. It is this latter driver which Paragon intends to replace.

      “The need for a new native implementation included with the kernel comes as the current NTFS driver remains practically unmaintained, lacks decent write support and has none of the other advanced features,” Paragon said. Its driver is not only read/write but supports additional features including journal replay, compressed and sparse files, and more."

      1. Anonymous Coward
        Anonymous Coward

        @Boothy

        Cant' agree with you more about the kernel driver. Assuming it is the same as the one in MacOS, you can make it to do read/write But it's a faff and the result is so half baked as not to be worth it.

        Bought the Paragon thing a while back. Once again, making the assumption that it will be same as macOS, it really works so well, it is invisible.

        1. Soruk

          I evaluated the Paragon driver for my then employer, and back then I wasn't entirely impressed. Sure, it was faster than NTFS-3G but it's error checking left a lot to be desired. Point it at a non-NTFS partition? Kernel panic. Hit a bad sector on your hard disc? Kernel panic. We ended up running with NTFS-3G as running slower was preferable to collapsing in a heap at the slightest hint of trouble.

    2. devin3782

      Basically it'll come with a massive write performance boost and god knows it needs it the fuse driver is horribly slow, very much looking forward to using this.

    3. bofh1961

      What's the use case for NTFS?

      If you're not using Windows why would you want it? I used to use the FUSE driver when I had a dual boot machine but haven't had a need for it since. Now, exFAT support is useful.

      1. MattPi

        Re: What's the use case for NTFS?

        In my personal use, being able to plug in external disks that move between my partner's Windows laptop and my Linux box. I don't think that's too unusual for big data sets that network (SFTP, etc.) or shares would be cumbersome.

        Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

      2. ThatOne Silver badge

        Re: What's the use case for NTFS?

        > If you're not using Windows why would you want it?

        I know I will be downvoted for admitting it, but the vast majority still uses Windows, and unless you are isolated and only communicating through Internet, NTFS support is necessary for all your shared resources (portable HDs, network shares, etc).

        (Now I agree full exFAT support would be great.)

        1. eswan

          Re: What's the use case for NTFS?

          NTFS is in no way needed for network shares, and is unecessary for almost all shared resources.

          1. Anonymous Coward
            Anonymous Coward

            Re: What's the use case for NTFS?

            Depends if you have to worry about preserving file permissions, and don't want to deal with the constant hell of DIY metadata preservation.

            I for one look forward to a glorious future of mounting NTFS volumes R/W on linux servers, as it will be a boon for malware analysis and remediation, backups and restores, disk imaging, etc.

            It might also be fun to be able to mount drive image files on a linux server that presenting them as iSCSI volumes. Once the code has it's bugs shaken out it would be wonderful to mount, de-dupe, compress, and backup volumes for incrementals, stuff like that.

            1. Anonymous Coward
              Anonymous Coward

              Re: What's the use case for NTFS?

              “ It might also be fun to be able to mount drive image files on a linux server that presenting them as iSCSI volumes. Once the code has it's bugs shaken out it would be wonderful to mount, de-dupe, compress, and backup volumes for incrementals, stuff like that.”

              Fun? I hope I never meet you at a party!

              1. Wayland

                Re: What's the use case for NTFS?

                Always in the kitchen at parties.

        2. katrinab Silver badge
          Boffin

          Re: What's the use case for NTFS?

          Portable HDs, yes; but network shares use Samba.

        3. bofh1961

          Re: What's the use case for NTFS?

          For the most part I do only move data through the internet. I wouldn't quite call that isolation. The only other machines I share physical media with are also Linux so I guess I'm lucky there. I'll just have to accept that enough Linux users need it to make it worthwhile. In the meantime, I'm still wondering if MS will buy out Canonical...

      3. rcxb1

        Re: What's the use case for NTFS?

        <blockquote>If you're not using Windows why would you want it?</blockquote>

        Because other people use Windows, and on occasion it's useful to exchange files with them.

        The best way to exchange files is actually creating a properly formatted UDF disk, but that takes a bit of effort to deal with the bugs/limitations of Windows & Mac. This script makes it easy: https://github.com/JElchison/format-udf

      4. Binraider Silver badge

        Re: What's the use case for NTFS?

        I don't use Windows at home, but NTFS is impossible to avoid on work devices. Occasionally found on portable storage too (for better or worse).

        This might sound dumb, but I also have a large data drive. An out of the box linux distro, from a user perspective, generally doesn't know what to do with all the other disks that sit in your system. They know they are there, sure, but don't know what to do in terms of sticking a label and mount point in place for them. This is, IMO, a big downside to linux desktop usability. It's often bad with USB sticks too for same reasons.

        NTFS happens to be leftover on my data drive, because legacy of what the disk was previously used with (Win 7). The content is backed up to tape, but I haven't reformatted to Ext4 at this point. Also, getting my tape drive to work in Linux is an absolute ballache in anything other than RedHat or Centos 7; neither of which I particularly want around as daily drivers.

      5. Wayland

        Re: What's the use case for NTFS?

        I presume it would help when presented with a removable drive. You might not be the person choosing the disk format.

        1. Anonymous Coward
          Anonymous Coward

          Re: What's the use case for NTFS?

          Not a problem. Read the files, reformat the removable drive to EXT4, copy the data back, and give it back to the asshat who handed you a drive with a defective filesystem.

    4. eldakka

      Currently, if you want read/write to an NTFS filesystem you need to either use the very slow FUSE implementaton, or go and find 3rd-party (proprietary) kernel modules and install (and possibly compile) them. Paragon is one of those 3rd-parties who provides an NTFS driver.

      This is Paragon open-sourcing and GPL'ing their driver and putting it into the Linux kernel as a standard kernel featureset. Therefore once this is properly incorporated, then out-of-the-box as it were, the Linux Kernel will support read/write to an NTFS filesystem just like any other kernel filesystem driver, i.e. blazing fast compared to the user-space FUSE implementation, and without having to go out and find a 3rd-party driver.

      1. Anonymous Coward
        Anonymous Coward

        Thank you for your answer. Not sure why I got downvoted for asking a genuine question...

      2. Sanguma

        ... as a wet week

        It's a job-and-a-half backing up from a distro like Kubuntu when you're backing-up onto a portable NTFS drive. It takes quite a long time, and you can guarantee that when you connect it to a MS Windows machine, it'll tell you there are errors that must be corrected.

        The sooner the better this gets fixed. It can be a real pain at times.

    5. oiseau
      WTF?

      ... what this driver brings to the table that we don't already have.

      Beat me to it.

      Quite so ...

      Just why does the Linux kernel need to add ~27K lines of code to the kernel?

      Is this really necessary?

      To get exactly what in return?

      From where I stand, I see absolutely nothing.

      It's much too little, far too late and with excess caveats attached.

      As far as I know, Linux currently has two NTFS drivers, a FUSE (Filesystem in Userspace) driver which is read/write, and a kernel driver which is read-only.

      I've managed quite well with that when I have needed to look at/write to the odd HDD with a NTFS filesystem and in 10+ years using Linux, I really cannot remember the last time I accessed a NTFS partition.

      Adding 27K lines of code is nothing short of a sure recipe for serious trouble further on.

      ---------------------------------------------------------------> Timeo Danaos et dona ferentes <---------------------------------------------------------------

      O.

      1. Androgynous Cupboard Silver badge

        Why stop there? I imagine if we assembled a list of everything you haven't used and don't see any need for we could trim the kernel by quite a bit.

        1. Solviva

          Thats the beauty of make (menu)config. Pick the bits you don't want/need.

        2. vtcodger Silver badge

          Custom kernel

          Aside from which if one doesn't want certain stuff in their kernel, it didn't used to be all that hard to build a custom kernel without the thing(s) one doesn't want. Presumably still could be done if one cares enough.

    6. jetjet

      You can fix a botched Windows by mounting the hardrive in Linux.

      * For example missing files/.DLLs

      * Reset forgotten Windows administrator password - easy fix if you access the disk via Linux.

      * There are even tools in Linux to edit Registry

      * Cleanning Windows from viruses under Linux.

      * Data recovery if Windows doesn't start properly, or files are locked etc.

      * Windows backup/restore.

      * Forensics

      * Studying Windows

      I used to fix Windows which loops in reboot - marked Windows file system as dirty under Linux (there is a tool for that). Next boot windows fixed it by itself and booted Windows fine.

  2. Anonymous Coward
    Anonymous Coward

    About the only real-world use-case (I can think of) is support for mixed environments where you don't want end-users to care what format they selected on their thumbsticks. And that's not something Karen in reception screws up twice.

    1. Wayland

      Well obviously! Mixed environments.

      NTFS is probably the largest environment for files on this planet.

  3. Skiron

    Now that MS appear to be Linux lovers, why don't they donate the code/fixes - they are supposedly the experts on this?

    1. Anonymous Coward
      Anonymous Coward

      This may be Paragon trying to get out ahead of MS doing exactly that and rendering Paragon completely irrelevant in the Linux file system space. If they’re the authors of the official in kernel Linux NTFS implementation then third parties will still have a reason to go to Paragon if they want to pay for a custom feature or other optimisation.

      1. DS999 Silver badge

        There is no guarantee of that

        Paragon's driver can be added but if Microsoft releases the "real" NTFS for Linux next year it would also be accepted into the kernel. It would be up to the distros which one they enable, but safe to say it would probably be Microsoft's.

        I mean, Linux already has two NTFS drivers, the current in kernel one that is read-only, and the pretty full featured (from what I've read, even more full featured than Paragon's) that runs in userspace using FUSE.

        1. Wayland

          Re: There is no guarantee of that

          Paragon is known for specialising in this area so their driver could be better than Microsoft's. However Microsoft would be able to carry over the bugs and quirks flawlessly.

  4. Anonymous Coward
    Anonymous Coward

    I'm surprised so many people are against this, or downplaying the importance of this being merged in..

    NTFS is a major filesystem, and for the longest time has been annoying, cumbersome, slow sand sometimes even dangerous to deal with from within Linux.

    An in kernel driver for NTFS that provides safe rw support is huge.

    1. FIA Silver badge

      I'm finding the commentary fascinating TBH.

      From my point of view: fairly well known company open sources (good thing) their previously closed source (ie, seen as a bad thing in the Linux world) driver.... to a sea of complaints. :)

      Now whilst Paragon may be doing this for selfish reasons (that's how most businesses work), it still won't have been an insignificant effort on their part (assuming they have done all the legal dilligance and it's not just one roge dev team). They also do seem to have engaged with the (sometimes quite fearsome to onlookers) linux kernel devs too.

      Seems you really are damned if you do and damned if you don't.

      1. Anonymous Coward
        Anonymous Coward

        As an optimistic pragmatist

        I welcome the ability of one of my platforms of choice to be able to link to the default file system of my main platform of necessity.

        Others admins may be grouching because of years of bad blood with the great satan from Ye Olden Tymes, or becuase they are operating a pure linux enviornment instead of a mixed shop. People tend to fall into that I don't need it so it must be stupid mindset as often here as everywhere else.

        I hope M$ is actually involved in this, as once Paragon is though it's teething problems dealing with Linus, a few M$ coders could iron out any kinks left in Paragon's implementation faster then re-implementing it over again, and the FUSE team can backport changes from the kernel if they come back out of hibernation.

  5. Jesse.Bacon
    Linux

    What For?

    First of all, it’s Linus. I want people verifying he had anything to do with this. NTFS is not a thing I need support for where I use Linux. Alternate data streams are just not a priority for my environment. Due to the transparency of text based documents I am good to go with file hashes.

    1. Pascal Monett Silver badge

      Re: What For?

      We're absolutely thrilled that you have everything you can possibly need.

      We shall forthwith call all developers and have them stop whatever it is they're doing, since you don't need it.

      1. Anonymous Coward
        Anonymous Coward

        Re: What For?

        Yep, there are lots of things in the Linux kernel that I don’t need. I demand that they are all removed forthwith.

  6. Claverhouse Silver badge
    Meh

    Fuse works seamlessly on KDE

    Dolphin has many faults, but a couple of hard drives are NTFS for reasons of sharing with some people stuck with Windows *, and I've never come across a file it won't open.

    .

    .

    * The bugger is when the drive goes off and screams for chkdisk, and one has no Windows machine to try it on. Tried WINE, but downloading that utility has problems of its own...

    1. rcxb1

      Re: Fuse works seamlessly on KDE

      <blockquote>Fuse works seamlessly on KDE</blockquote>

      Works, yes but it is very slow. You'll notice if copying large files. Glad ntfs-3g exists, but I look forward to the improved kernel driver.

      <blockquote>The bugger is when the drive goes off and screams for chkdisk, and one has no Windows machine to try it on.</blockquote>

      ntfsfix on Linux usually gets it going.

      These days, Windows is free to use. You can download the ISO from microsoft.com, tell it you don't have a key, and run it indefinitely. You can create a portable version installed on a USB drive if you don't have a place to put it, or just grab a copy of HBCD: https://www.hirensbootcd.org/download/

      1. David 132 Silver badge
        Thumb Up

        Re: Fuse works seamlessly on KDE

        Upvoted just for mention of HBCD. God, is that still going? I remember X years ago when Hiren was my go-to resource for fixing Windows machines. ISTR it was legally a little dodgy at the time - lots of the mega-useful software included on it was actually full/licensed versions. Did it go legit?

  7. AlvordSky

    Which NTFS features will be usable?

    Is anyone aware of which NTFS features will be accessible from Linux - or will translate into something Linux understands? For example links and reparse points, file date/time granularity, SID translation, NTFS ACLs reversiblly mapping to Linux/Posix ACLs (so I could fix the damned things occasionally), file name character restrictions (like colon ":"), alternate data streams (mentioned earlier), ...

    If there is reference to some documented intentions, please let me know!

    1. JBowler

      Re: Which NTFS features will be usable?

      >Is anyone aware of which NTFS features will be accessible from Linux - or will

      >translate into something Linux understands? For example links

      NTFS links == UNIX hard links. NTFS can hard link directories (producing a DAG; I don't know if there is any checking for complete cycles in the default implementation). UNIX file systems can do the same, but UNIX traditionally crashes when this happens (based on experience with a malformed UNIX FS).

      >and reparse points,

      Called "symbolic links" in BSD style Unices. Windows Explorer "shortcuts" won't work; they are a feature of the Windows "ls" command which specially interprets files with the suffix ".lnk".

      >file date/time granularity,

      Broadly compatible at this point I believe; the LCD might still differ (I don't know) but it doesn't matter in practice at present because they both have sub-ms accuracy.

      >SID translation,

      Nope, Linux canne do that captain. Remember that if you are using a particular file system as a native file system in Linux you are inherently constrained to using the OS'es identity mechanism. How well does Andrew work?:

      fs/afs: 21402 lines of .c

      >NTFS ACLs reversiblly mapping to Linux/Posix ACLs (so I could fix the damned things occasionally),

      The thing about NTFS is that it is a superset of the (then) available file systems; it's just like reiser4, you can effectively do anything. So you can put Linux ACLs into it and you can get Windows ones out of it, but the question you are asking is how to map Windows ACLs into Linux ACLs. That's not a question for the file system; that's a question for *you* (assuming you are not a file system.)

      > file name character restrictions (like colon ":"), alternate data streams (mentioned earlier), ...

      It's a Multics style file system, not a UNIX style one. That is true. A file can contain multiple date streams; not alternate ones, multiple ones. Like a file in MacOS (which has two - more than one - multiple), unlike a file in UNIX which both rigorously and religiously insists that a file is just a single bag of bytes (albeit ordered; they never mention that!)

      So? Linux can never be MacOS - it only has ONE stream in each file - and it can't be Multics and it can't be Windows. But colons? Seriously? It's convenient to have a *stream* delimiter that is distinct from a *path name* delimiter but that is an OS consideration. The syntax of a path name, including one with files that have multiple streams, is determined by the OS, not the file system. This is why the ADFS file system works in Linux - it does *not* use the RISC OS directory separator! Likewise try using "\" as a directory separator in Linux when you have a FAT file system mounted.

      If Linus wanted to permit file**/**stream he could.

  8. JBowler

    It's still the Linux boys' club

    "Too big to review."

    fs/ext4: 55114 lines of .c

    fs/reiserfs: 28771 lines of .c

    fs/btrfs: 129905 lines of .c

    fs/adfs: 2374 lines of .c

    fs/fat: 7625 lines of .c

    So 27k for something that reiser4 was trying to emulate and has more that 20 years of consistent development history behind it and that, *just works*.

    1. James Hughes 1

      Re: It's still the Linux boys' club

      The file systems you quote have been in development for many many years, they were not introduced to the kernel as monolithic chunks of code, as happened in Paragons case. They have gestated over many years and many commits, each (mostly) small enough to be properly reviewed.

    2. Anonymous Coward
      Anonymous Coward

      Re: It's still the Linux boys' club

      Almost 130,000 lines for BTRFS? Yikes! That makes it bigger than the others all combined.

  9. badger31

    Dropbox support?

    I'd love to have my 2tb Dropbox account in one place only. My dual-boot machine currently has a Dropbox share on an NTFS partition for Windows and an Ext4 partition for Linux. Maybe that's just me

    1. Anonymous Coward
      Anonymous Coward

      Re: Dropbox support?

      Easy fix: Remove the Windoze partition.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like