Microsoft's only gone and published the exFAT spec, now supports popping it in the Linux kernel

Microsoft has published the technical specification for exFAT, a file system widely used for removable storage devices. Camera_and_microSDXC_card exFAT is widely used for high capacity SD cards in cameras exFAT stands for Extended File Allocation Table and is widely used for things like memory cards. It is the most recent …

[Dwarf]

What if ...

What if they only did this because they can see users moving off to other platforms and they needed a way for people to be able to bring data back to Windows machines via various removable media.

MacOS and Linux get on just fine without exFAT support, yet Windows doesn’t support any Mac or Linux file systems.

Nice to see Microsoft do something useful for a change too..

[Anonymous Coward]

Re: What if ...

macOS has supported ExFAT since 10.6.3.

[Nate Amsden]

Re: What if ...

you say that as if Linux systems haven't had exfat options for years when they have. I don't think it was in the stock kernel more likely via FUSE (see https://www.howtogeek.com/235655/how-to-mount-and-use-an-exfat-drive-on-linux/ )

I think it was probably a much bigger issue for android devices at least those with SD card support. Lots of old news stories about OEMs having to pay lots of licensing $$ to MS for Android phones. Though as far as I can recall those agreements were always secret, don't know if details ever came out about what specifically was being licensed.

Though I'm sure native exfat in the kernel will provide much better performance than userland drivers.

by contrast getting exFAT to work in 32-bit XP is far more difficult(I did it a couple of years ago setting up an XP system to play older games ended up losing interest in games again). There was a patch (KB955704) MS released but have long since removed from their site(don't know why, 64-bit patch is readily available), though there are folks out there that kept copies(including me now).

after reading the GIMP article earlier(and thinking back further to the master-slave stuff from python was it??) I'm sort of waiting to someone to come out and be offended by the word "FAT" maybe say something like "exFAT is saying I was fat before but not anymore or something".

(as a fat guy I am not offended, by just about anything)

[Anonymous Coward]

Re: What if ...

The would probably consider it misogynous and giving unrealistic view on what beauty is since exFAT is, you know your ex partner is fat. Cause you know its only the men that have an unrealistic expectation of beauty.

(dadbod here and no shame)

[Fatman]

Re: What if ...

<quote>I'm sort of waiting to someone to come out and be offended by the word "FAT" maybe say something like "exFAT is saying I was fat before but not anymore or something".

(as a fat guy I am not offended, by just about anything)</quote>

As another EXFAT guy, nor am I; but we must think of the snowflakes....

PC run amok!

[joeW]

Re: What if ...

Have you tried turning it off and on again?

[Tigra 07]

Re: What if ...

The future implementation to support file sizes over 4gb is to be called EXOBESE for unrelated reasons...

[the spectacularly refined chap]

Re: What if ...

It already does, it's the principal benefit over the older FAT formats.

[Anonymous Coward]

Re: What if ...

...yet Windows doesn’t support any Mac or Linux file systems.

Apple have provided read only HFS support for Windows for yonks.

Windows itself is perfectly open to supporting any old file system provided someone is willing to write the drivers. There are commercial solutions for ext file systems on Windows. AFAIK no one has seen fit to do an open source driver for ext on Windows...

[Ilgaz]

Paragon software may have it

I think they have commercial software doing the thing you described, pluggable filesystem.

[Scoured Frisbee]

Re: What if ...

> AFAIK no one has seen fit to do an open source driver for ext on Windows...

https://sourceforge.net/projects/ext2fsd/

Been using it recently for pulling data off a USB ext2 drive, but I seem to recall having previously used it with ext3. Can't speak to its writing but reading is great.

It is a user space reader, I'm not sure if there's a way to make it load dynamically when a drive is connected. It's been pretty stable, and I don't reboot very often, so this has not been a huge limitation for me as a technical user - for my wife and kids it would be, but then they have no interest in reading my old backup drive anyway...

[Mage]

Re: What if ...

There are free ext drivers for XP, at least ext2

Wasn't FAT based on one of the hundreds of CP/M formats. As DOS itself was a rip off of CPM/86.

The additions to fudge exFAT shouldn't have been permitted as patents. Only the actual code should have been copyright. It fails almost every traditional test, as do most patents at the USPTO, because they get paid more for acceptance than rejection and the theory is that if the patent is invalid someone will take the holder to court. The system favours huge corporations that spend as much on patent lawyers and clerical staff as actual engineers actually researching and developing real products. The system is broken.

However try reading an NTFS formated USB drive on a locked down Education institute Mac, or an iOS thing on Windows or Linux without iTunes.

[Richard Plinston]

Re: What if ...

> Wasn't FAT based on one of the hundreds of CP/M formats.

No. There were not "hundreds of CP/M formats". There was one file system implemented on many different capabilities of hardware. For example drives may have been hard sectored or soft sectored, FM or MFM, single density, double density, quad density, 40 or 80 track. 8, 9 or 10 sectored, with various skew factors. These different types of hardware were chosen for various cost, performance and availability reasons. For example particular controllers were slower and required a longer time between sector reads and thus a particular skew factor and/or inter-sector gap.

Even with [MS-]DOS there were still 'dozens of formats'. The original diskettes on an IBM PC were 160Kb. Other companies used different controllers and drives and could not directly exchange data with different machines. PC-Alien could read and write many 'alien' CP/M formats _and_ many 'alien' MS-DOS formats.

It was only when all manufacturers started building IBM clones that they (mostly) used the same drives and controllers and became compatible with IBM PC diskettes.

But, no, FAT file system in the way it allocates and records file sectors is completely different from CP/M's. The only similarity is part of the directory entry so that the FCBs can be compatible for converted software.

> As DOS itself was a rip off of CPM/86.

No. It was a rip off of CP/M converted to 16bit 8086/8088. It was probably CP/M 1.3 because PC-DOS 1.0 had a bug in the FCB handling on a close that existed in that version of CP/M. Both Microsoft and SCP were OEMs for DRI CP/M (MS for their Z80 softcard) and had all the code that DRI gave to OEMs.

[jelabarre59]

Re: What if ...

Back when MS first came out with exFAT, I thought it would have been a smarter idea for the various consumer electronics manufacturers to get together and decide on an open spec like EXT4, or something else that was already capable of handling larger storage capacities. MSWin already had it's IFS (Installable Filesystem) support, so all a manufacturer would have to do is include the driver disk with the device (along with the shit-ton of other crap they usually bundle, withe added advantage taht once it was available for one device, it would be available for ALL of them.

But that would have required them to NOT be a bunch of lazy, chickenshit bunch of cowardly lemmings.

[NoneSuch]

Re: What if ...

"Nice to see Microsoft do something useful for a change too.."

MS are members of the Linux Foundation (among other major corperations, see link below) and have way too much say in Linux development IMO.

Let's hope the legal eagles check out the boilerplate carefully before the Ex-FAT code is amended or MS may very well be putting a poison pill into the OS.

I don't trust Microsoft and their "embrace, extend, and exterminate" motto.

https://www.linuxfoundation.org/membership/members/

[werdsmith]

Re: What if ...

MS are members of the Linux Foundation (among other major corperations, see link below) and have way too much say in Linux development IMO.

Better start thinking a new name soon. Winux? Lindows? Or maybe just WindowSUX ? WInSUX?

[phuzz]

Re: What if ...

"I don't trust Microsoft and their "embrace, extend, and exterminate" motto."

Microsoft are a massive company, and while I'm sure there's forgotten departments still fighting against OS/2, there are entire divisions who are genuinely happy to contribute towards open source software. The main problem is that the left hand often doesn't know what the right is doing, and is quite possibly working in a completely different direction.

It's a bit like Sony in the late 90's, trying to push the Minidisk (which was super handy for pirating music), whilst also having a recording company who were fighting against piracy at every turn, both at the same time.

[AIBailey]

Re: What if ...

It's a bit like Sony in the late 90's, trying to push the Minidisk (which was super handy for pirating music), whilst also having a recording company who were fighting against piracy at every turn, both at the same time.

SCMS took care of the piracy issue, meaning that digital copies made on consumer machines could not then be used to create additional digital copies, meaning that analogue -> analogue was the only way to create a further copy.

It made pirating of music no easier than using good ol' cassette tapes, though obviously the quality was (at least in theory) better.

[jelabarre59]

Re: What if ...

The main problem is that the left hand often doesn't know what the right is doing...

I think the line from "The Rutles Story" was "...didn't know WHO the right hand was doing..."

[Anonymous Coward]

Re: What if ...

You fail to notice exFAT is especially used on imaging devices like photo cameras and video cameras. Maybe you think Linux doesn't need to 'bring back' data from those devices?

[Anonymous Coward]

Re: What if ...

We've got collaborators putting TB of stuff onto drives for us, from a pretty high-bandwidth source. They say they found they need exFAT for this (no idea if I believe them and no way to argue), so we have to get data from exFAT onto a Linux system where it is to be analysed.

I'm well aware that we can install the FUSE packages as they have casually suggested, but while I feel fairly confident that MS is unlikely to sue me for doing so on my own machine (outside US where the patents may or may not apply), I'm less confident about the conversation I might have to have with any internal auditor. (Assuming one exists who understands Linux repository naming... actually it's going to be fine isn't it? Still, would be nice to see the patent indemnity there for this kind of thing. Obviously there are workarounds available to us, sadly they're tedious.)

[doublelayer]

Re: What if ...

While they don't necessarily have to use exFAT for that, it is probably the logical choice. ExFAT is a filesystem that plays very well with external media and most operating systems, Linux included if the package is installed. Most other filesystems don't meet one or both of those requirements. If they're also using Linux, you have many other options including the basic EXTs, but that will be more trouble for them than it's worth if they use Windows or Mac OS. Of course, the code installed to let Linux deal with exFAT at the moment won't be all that fast, which a kernel implementation would fix. I am therefore quite hopeful that this does get implemented soon.

[Anonymous Coward]

Re: What if ...

My preference would really be NTFS, as there are no patent issues (or at least, I can install NTFS from EPEL which at least means RedHat is happy to distribute it) and they seem to be using Windows.

[bombastic bob]

Re: What if ...

actually Mac and Linux _DO_ have exFAT support, via fuse

it's been that way for a while. and the fuse drivers are pretty good. It's a little klunky but works fine (and also in FreeBSD, worth pointing out).

What I'd like to see: universal userland FUSE support in WINDOWS - WITHOUT the need of SIGNED DRIVERS.

[phuzz]

Re: What if ...

Signing drivers on Windows isn't terribly hard (you do need a cert from a trusted chain, but that's not too difficult to obtain) as long as you don't need kernel-level drivers, and even then it doesn't look too tricky.

Microsoft are stuck between allowing people to install whatever drivers they can download, which will result in people installing junk/malicious dirvers, and then blaming Microsoft when it breaks. Or they can try and lock down what drivers can be installed on Windows, and hopefully keep it more stable and secure, but have to put up with people complaining about driver signing.

It's a fine line, I'd be happy with a more convoluted way of disabling the protection, so that technical users could run whatever drivers, but it would be impossible for the average user, but having seen enough spam emails that walk the recipient through enabling macros, I worry that might be a stop-gap at best.

[Piro]

Re: What if ...

MacOS wouldn't get on very well if it didn't support exFAT, seeing as it's used by people with shiny cameras and such, where the default format for large cards is... exFAT. Of course it supports exFAT. It has done for 10 years.

[Joe Montana]

Re: What if ...

Microsoft intentionally don't support any other filesystems, and there are several which are designed specifically for flash based storage. Exfat is by no means the best option, the one and only reason it's widely used is because it's the only option supported by windows.

[TheVogon]

Re: What if ...

Other than native support for:

FAT

FAT32

ExFAT

NTFS

REFS

and many others such as BTRFS via 3rd party drivers you mean?

[Anonymous Coward]

Re: What if ...

That's not many, and certainly none of those are optimised for removable flash storage (ExFAT is arguably the closest).

Third party drivers don't count as removable media needs to be supported out of the box.

[jelabarre59]

Re: What if ...

What about ReiserFS? I've heard it's a killer filesystem...

[Anonymous Coward]

Re: What if ...

If MS was bothered about other users getting their data into MS platforms, they'd just port one of the other systems filesystems, maybe restricting it to sdcard use only.

[Anonymous Coward]

Patents about to expire ?

[Tom 7]

2006 I think - I thought they had 20 years.

[doublelayer]

A quick check that might be incorrect suggests that they were granted a central patent in 2009 source, and then the patent would expire in 2026. I'm wondering why the patent was granted in 2009 when the filesystem was released in 2006, so there are some other options. But no, it seems they could keep charging for the patent for a few more years, and if they release it under acceptable terms, they're not going to do so.

[Charles 9]

It may have taken three years for the patent to actually be granted, and the clock starts from the filing date.

[TheVogon]

I wouldn't think ExFAT revenue from Linux is significant

I think it costs 4 quid at retail to licence ExFAT on a Synology NAS for instance. Microsoft can easily afford to drop charges for Linux and that might even drive increased 3rd party licencing revenue.

Much better to do that and retain a non Linux licencing revenue stream for ~ another decade than risk it being replaced as many millions of Android devices already don't support ExFAT for instance.

[Maventi]

You nailed it. This move is great PR plus it means exFAT support will become standard issue in Android, further cementing it as a standard.

Android vendors mostly don't bother at the moment as it's currently an extra cost that eats into already thin margins. The completely proprietary implementations (e.g. cameras) still need to pay.

It's also low risk; at this point MS have well passed recovering any R&D costs involved in developing the format, and with cloud services bringing in the real revenue it's not really a big deal if the proportionately small income stream from exFAT starts to dry up a bit.

[ecofeco]

It's a trap!

Don't fall for it.

Linux should never have the taint of MS on it or in it. It would be an abomination.

[Charles 9]

Re: It's a trap!

Too late. You're forgetting NTFS support.

[Updraft102]

Re: It's a trap!

And SMB...

[DuncanLarge]

Re: It's a trap!

SMB came from IBM in the early 80's way befor MS got their hand on it, buy buing it.

Also SMB support on GNU/Linux comes from Samba, a completer re-implementation of the protocol. No MS involvement.

[Rockets]

Re: It's a trap!

And Hyper-V drivers

[Saint]

Re: It's a trap!

And MS SQL Server

[Alan Bourke]

Re: It's a trap!

Yeah because then it would run desktop software that people actually use and we can't have that.

[Pascal Monett]

Is Microsoft really that desperate ?

Or is MS actually starting to change ? Could it be that enough Linux-friendly people have been hired, or is it just the fact that MS sees sooo many Linux servers, phones and slabs as is starting to realize that its citadel is becoming its tomb ?

In any case, I think I noticed a pig with wing stubs today.

[hitmouse]

Re: Is Microsoft really that desperate ?

When Microsoft bought Hotmail over 20 years ago it became one of the biggest BSD/Linux managers in the world. It also released its first Linux kernel last year.

The important thing with IT companies is not to listen to the hot air released at the top (particularly during the Ballmer era) but pay attention to what the engineers are actually doing.

[Henry Wertz 1]

Re: Is Microsoft really that desperate ?

"When Microsoft bought Hotmail over 20 years ago it became one of the biggest BSD/Linux managers in the world."

Temporarily. I remember stories from the time, where they almost immediately began moving hotmail from BSD to Windows Server; they ended up I think doubling or tripling the number of servers to keep performance usable. Partially because BSD at the time was simply far more efficient than Windows Server. Partially because they apparently just kind of stuck the Hotmail software stack right on Windows, and it used the typical UNIX server design of forking new processes (per connection or user?) whereas Windows (at least at the time) performed far better if you used threads rather than processes.

[NogginTheNog]

Re: Is Microsoft really that desperate ?

Hotmail has run on Exchange for a long time.

Where do you think the Exchange Online/cloud stuff got developed and tested..?

[Updraft102]

Re: Is Microsoft really that desperate ?

Microsoft isn't really all that concerned about Windows anymore. If it were still the be-all and end-all of Microsoft's vision, they would not be treating it and its users as they have.

Meanwhile, while they're busy not caring about Windows, MS is raking in masses of cash with Azure, which has more Linux VMs than Windows VMs.

It's not that MS has become enlightened and now likes the idea of free software. It's just that Linux is not a competitor to their main product anymore, but is instead a route to making money. Their main products are in the cloud, and the cloud runs on Linux. That's why they claim to love Linux... which maybe they kinda do, given that it's making a pile of money for them.

If MS was ever going to move to EEE Linux, it would have been years ago, when Linux was actually a competitor with something MS cared about. Now it would be foolish to try to kill off something that has them making as much as they are.

[Joe Montana]

Re: Is Microsoft really that desperate ?

MS treat its users the way they do because they can.. When users are locked in you can treat them however you want and there's nothing they can do about it.

You constantly hear people complaining about MS, and yet they still remain customers. Until significant numbers of them start becoming ex customers, they have no reason to change.

Similarly the company is unlikely to change, no doubt EEE is still their strategy... If you're running a linux vm in azure, it could just as easily become a linux vm in aws - MS have never been able to compete in an open field, so they will be looking for some way to lock customers in here too.

[Morten Kristiansen]

Bring compatibility problems to Window, not the other way around

Why not implement something like EXT4 in Windows and a stack for embedded devices. Removable medias should have a small partition with the Windows stack in front so users can install when they insert the media.

One of the reasons OS/2 died was because it was compatible-ish with Windows. It ended up as an inferior Windows. Lets move the problems with drivers to Windows.

[Anonymous Coward]

Re: Bring compatibility problems to Window, not the other way around

OS/2 usually ran windows app faster and better than windows did.

It was because it required better hardware to run well, plus it didn't get a massive amount of development for it because why develop for OS/2 natively when you can just write for windows and both OSs can run it.

I remember running warp on my 486dx2 66 (16MB RAM)I believe it was (might have been the 386sx 25 (4MB RAM)) could run videos smoothly with multiple applications running on OS/2, whereas the same video could barely run on Win 3.1 on the same machine with nothing else running.

[Updraft102]

Re: Bring compatibility problems to Window, not the other way around

One of the reasons OS/2 died was because it was compatible-ish with Windows. It ended up as an inferior Windows.

It ended up being perceived that way, but that doesn't mean that's what it was. I never used OS/2, but it's hard to imagine anything being inferior to Windows in that time period.

The example of OS/2 deciding to become Windows compatible and thus sealing its own fate is cited fairly often, particularly when the topic is something like whether Windows phones should have been made to run Android apps or something like that. I don't personally think the decision to make OS/2 run Windows software sealed OS/2s fate as much as it failed to rescue it. OS/2 was already in trouble when they decided to make it run Windows software... that was why they made it Windows compatible. It overcame the disparity of software issue, the same thing that sunk Windows phone and prevents Linux from gaining more market share on the desktop, but people perceived it as a knock-off of the "real thing," which was readily available to them at an attractive price.

The very PCs that were running this Windows (3.0 at the time) were mostly IBM "clones," as they were derisively called, which had eaten IBM's lunch by providing the same hardware at a better price. IBM had to be perplexed that Compaq and other companies had made a lot of money being compatible with the IBM PC, yet when IBM made OS/2 Windows compatible, it just cemented Windows as the standard by which others are judged.

[Henry Wertz 1]

Re: Bring compatibility problems to Window, not the other way around

"The example of OS/2 deciding to become Windows compatible and thus sealing its own fate is cited fairly often, particularly when the topic is something like whether Windows phones should have been made to run Android apps or something like that. I don't personally think the decision to make OS/2 run Windows software sealed OS/2s fate as much as it failed to rescue it. OS/2 was already in trouble when they decided to make it run Windows software... that was why they made it Windows compatible."

Just wanted to back you up on this; you are correct sir. I ran OS/2 a bit back in the day, and indeed by the time later OS/2 versions came out with Windows compatibility (actually just including a copy of Windows and modifying it to run it's stuff seamlessly on the OS/2 desktop), by then OS/2 was basically down for the count. It probably increased their sales above the approximately 0 it would have been otherwise but I doubt by much. ("Approximately" 0 because there's apparently still some ATMs and such using it...)

[big_D]

Re: Bring compatibility problems to Window, not the other way around

Also Microsoft and IBM played cat-and-mouse for a while, with IBM giving OS/2 Windows compatibility and running Windows applications, then MS would change something in Office and the "key" applications for most users would stop working in the latest version, then IBM fixed that trap and the cycle started again.

[Anonymous Coward]

Re: Bring compatibility problems to Window, not the other way around

I found more issues with non-MS applications than MS ones. I could run Office without issues, but Borland Delphi 1 didn't run properly, nor some Lotus application - despite IBM having bought it.

Anyway, OS/2 never supported 32 bit applications, so when Windows 95 and NT 4 were out, and application were fully ported to them, OS/2 was left with older 16 bit ones.

[Richard Plinston]

Re: Bring compatibility problems to Window, not the other way around

> Anyway, OS/2 never supported 32 bit applications,

OS/2 supported OS/2 32 bit applications.

Windows 3.11 on OS/2 could run 32 bit applications when the win32s add-on was installed, exactly like the usual Windows 3.x. However, Microsoft added a particular feature in a later version of win32s that did a memory access beyond the 2Gb virtual memory limit of the OS/2. The only purpose of that was to stop the latest version running on OS/2.

"The job ain't done 'til Lotus won't run!"

[Richard Plinston]

Re: Bring compatibility problems to Window, not the other way around

> OS/2 never supported ...

There were other Microsoft tricks that were done to break OS/2:

* OS/2 required about 22 diskettes. When a new release of OS/2 was due Microsoft went to the few diskette manufacturers and bought the next six month's supply of diskettes. These were loaded with 'Windows for Workgroups 3.11' and stored in several warehouses awaiting sale as it was around a year's supply. Then MS announced Windows 95 and sales of WFW dropped. Eventually this became known as 'Windows for Warehouses' and millions of diskettes were dumped. It did kill OS/2 sales though which is what was intended.

* IBM was allowed to use any _current_ version of Windows. When Windows was originally to be included in OS/2 the current version was 3.1. Microsoft announced that 3.11 would be released 'soon' so IBM built the new OS/2 with 3.11 and announced it. The anti-monopoly ruling required that IBM could not pre-announce products more than 3 months ahead so there was a date that they had to release by. Microsoft held off the release of 3.11 until after that date so that OS/2 had to be released with the old 3.1.

[doublelayer]

Re: Bring compatibility problems to Window, not the other way around

I'm completely fine with you implementing EXT4 support for Windows. But unless it gets installed by default, it's useless for most cases. The average user isn't going to understand that they have to open the partition they see, install the driver, then remove and reinsert the media, probably after restarting, and then it'll work. That will just annoy them. And most people who have EXT4-formatted removable media aren't going to bother partitioning it to include the drivers for Windows, and for that matter Mac OS as well.

For removable media, I want the guarantee that I can plug my disk into anything, and the files will be there without needing to deal with drivers, request extra access to install them, or require reconfiguration. We already have a thing that does that, and it's FAT. The only tiny problem with FAT is that it contains a couple very irritating defects, the most obvious of which is the limit on file size that can quite easily be exceeded. But because we don't have anything else that pretty much every operating system understands, I still use it for most of my removable media. Getting a better version that doesn't have those defects and having that run on everything new would be wonderful. I don't really care which particular filesystem it is; if everyone adopts EXT4 I'd be equally content. But it's got to be built in.

[fwadman]

Re: Bring compatibility problems to Window, not the other way around

Because you don't need EXT4 on a memory stick. On a memory stick you want the smallest possible overhead file system and no permissioning. exFAT suits this very well. Why do you want to waste space on my memory stick with another partition with drivers (which will be both out of date .. and also a virus writers delight)

[Dan 55]

Re: Bring compatibility problems to Window, not the other way around

UDF also suites this very well but it was NIH.

[bombastic bob]

Re: Bring compatibility problems to Window, not the other way around

FuseFS for windows would solve this. But ONLY if the file system drivers DO NOT REQUIRE SIGNED CERTS [which Micro-shaft would NEVER approve of]. Since THEY are the keepers of the keys (aka STRANGLEHOLD) on that, no sneaking any drivers through that allow pure userland file systems, either.

[Anonymous Coward]

Re: Bring compatibility problems to Window, not the other way around

OS2 didn't die. It was killed.

Windows 95 wasn't finished until it ran in 4 GB of RAM (OS 2 wanted 5).

Result : loads of 16bit parts were kept, no proper process security, one dodgy driver would crash the whole thing.

[Peter Gathercole]

Re: Bring compatibility problems to Window, not the other way around @AC

Um. I think you may mean MB, not GB. At that time, GB was used for larger disks, and even then often only in single figures.

We had some 386DX systems originally running AIX/PS2 with 12MB, of RAM, and these ran OS2/Warp quite happily, and we even had the OS/2 versions of Lotus running without problems.

My first XP system only had 16MB of memory, and my first Windows95 system (in fact *my* first Windows system full stop) only had 4MB (before that time I used UNIX at work, and a BBC Micro at home, often as a VT100 terminal to the UNIX systems at work).

[Dave 126]

My uninformed comment

I've had issues around this, though I can't remember the details... something to do with trying to use USB OTG and an SD card reader on an Android phone to sort out photos taken on a compact camera. Seem to recall phone could read the card but not allow me to delete photos. Read only access. On holiday, no laptop available.

The point is, in 2016 I shouldn't have had such issues. I don't think it's unreasonable for someone to be able to buy an SD card, cheerfully start using it and then read and write to the same card with some other device. Uniformed? Quite right. This isn't the sort of thing someone should have to be informed about.

[rcxb]

Re: My uninformed comment

I don't think it's unreasonable for someone to be able to buy an SD card, cheerfully start using it and then read and write to the same card with some other device. Uniformed? Quite right. This isn't the sort of thing someone should have to be informed about.

I don't think it's unreasonable for someone to buy a petrol car and fill it up with diesel and expect it to run just fine.

You should direct your ire at Microsoft, as they invented exFAT specifically to enable them to collect patent fees on the "new" format, while the Linux bunch just decline to pay them. And secondarily device manufacturers, who gleefully jump into Microsoft's trap (taking you with them), when they could easily use a common and standard format like UDF (as found on DVDs).

[Phil Endecott]

Re: My uninformed comment

You should direct your ire at the SD card standards body, who in 2009 chose to require exfat support for the then-new SDXC cards, requiring everyone who wanted to support those cards in their devices (cameras etc) to license it from Microsoft, rather than choosing something freely-implementable like UDF.

[doublelayer]

Re: My uninformed comment

UDF is the solution? I've had a read through the wikipedia article about UDF, and I have my doubts. We'll start with the problem that it's designed for optical media. As in media that can be written a couple of times at most, not one that might have an operating system boot off it or store frequently changed files. There is a version not specifically designed for limited-writes media, but there are others designed specifically for that purpose. This brings us to the next point.

There are a bunch of revisions of the UDF spec. And we all know what that means: lots of poor implementations that support only some of them. And it's not just different release versions, but multiple types of filesystem inside UDF. The wikipedia page includes many statements about what versions different implementations support. Actually, they don't say that. They instead tell us what different implementations "claim to support". Sometimes, such a statement is followed by a statement that only certain subversions are correctly supported, and much of this is for reading only.

My third point can best be made with this quote: "The UDF specifications[7] allow only one Character Set OSTA CS0". When this is a key point in the summary of a spec, and they follow it with a discussion of when this doesn't play well with other encodings, I know it's not fun to deal too much with this filesystem.

[Dan 55]

Re: My uninformed comment

UDF revisions are highly backwards compatible (they have to be so people don't have to throw out their DVD players), there is an implementation of UDF specifically for flash media, and OS mplementations don't mean much from Vista onwards.

OSTA CS0 is compressed UTF-8 or 16 Unicode which at first glance doesn't seem an insurmountable problem.

[DuncanLarge]

Re: My uninformed comment

> We'll start with the problem that it's designed for optical media.

No its not. Its designed for a variety of media types. I use UDF a lot and can safely say that it has modes for optical and HDD. Perhaps you should read the page again and you will see that it has specific formatting modes for HDD's and anything that works like them.

Most of the development between versions focus on optical media because there was so much improvement to be had in handling wear levelling etc.

> As in media that can be written a couple of times at most

Lol, hundres of thousands of writes are a couple of times? I know of SSD's that can only handle a few thousand, what say you about those?

To summarise the rest of your comment: UDF is a file system that would have given us everything we would have needed. It was borked by the companies (M$) that want a monopoly on file systems use, with as you say, poor implementations. Thus, like other borked formats like RTF, UDF has basically been beaten up and left in the gutter to be used by those who need its features and dont mind dealing with the vendor implemented incompatibilities.

What other open and free filesystem can you think of? Its not FAT, yet thats on all SD cards and flash drives, its not NTFS, not HFS+. EXT(1,2,3,4), XFS, ZFS etc all meet this definition but guess what, you dont see them, even though they are superior, on large flash drives.

If this exFAT thing allows a Free SOftware exFAT driver to legally exist, maybe this will be a good thing.

[Anonymous Coward]

Re: My uninformed comment

Imaging devices has to write out data fast. Especially video cameras, but high megapixel cameras at high frames per second as well.

Very fast. And that often without too powerful processors, huge RAM buffers (which are still a liability) and without draining batteries quickly.

[rcxb]

Re: My uninformed comment

the SD card standards body, who in 2009 chose to require exfat support for the then-new SDXC cards, requiring everyone who wanted to support those cards

No, no, no...

First of all, the SD consortium only controls the trademark, so all they can do is keep you from putting the logo on it.

Second, there's no difference in SDHC vs SDXC hardware. The ONLY difference is the file-system. So it would have been idiotic to allow manufacturers to use the branding without doing the ONE THING that the branding indicate.

And why was the file-system changed? Because Windows 2000+ doesn't allow you to FORMAT a partition over 32GB as FAT32. You can stick a 2TB SDXC card in any old SDHC device, formatted to FAT32 (with Linux or Windows 98) and use it without any trouble in Linux, Mac, Windows, etc.

Sure, the SD consortium has a bit of sway, and they could have used it to TRY and oppose Microsoft instead of feeding into their monopolization attempts. But it doesn't make sense to blame them, when Microsoft is clearly the culprit.

[Charles 9]

Re: My uninformed comment

The other issue IINM is that FAT32 use beyond 32GB is discouraged by Microsoft so as to make people use NTFS instead (in hard drives, this is actually recommended as NTFS is more robust). Because of this uncertainty, using FAT32 beyond 32GB can get iffy. Third-party tools can format beyond 32GB, and Windows should be able to read it, but it's not officially supported: thus your problem.

[Richard Plinston]

Re: My uninformed comment

> The other issue IINM is that FAT32 use beyond 32GB is discouraged

I recall when FAT was limited to 32 megabytes. This was FAT16 of MS-DOS 3.x and limited to 16,000 (or so) clusters of 2,048bytes. Several manufacturers, such as Compaq and Gateway raised the limit in various incompatible ways. This made IBM uncompetitive because they stuck with the 32Mb limit until they rewrote the code to give a 128Mb limit and handed it back to MS to be MS-DOS 4.01.

The main problem with FAT though is that it is awful in handling large random access files, such as ISAM or databases. To access a particular sector it must start at the directory entry and read every FAT entry for that file until it gets to the required sector. This is very, very slow. Fortunately DR-DOS 3.x could format a disk with larger clusters. 8K clusters gave a 3x performance improvement on standard 2K clusters on ISAM files randomly read simply due to less FAT entries needing to be read for every step.

[Charles 9]

Re: My uninformed comment

The problem came if you had a combination of very large files and lots of very small files. This is because the larger cluster size made for simpler organization but also contributed to waste since the cluster was the smallest unit that could be allocated to a file, and if you have a bunch of tiny files, each one took up a whole cluster which would end up being mostly empty space. It's one of the tradeoff aspects of filesystem layout.

[Richard Plinston]

The original FAT

> The original FAT was used by DOS in the late 1970s.

There wasn't a "[MS-]DOS in the late 1970s". Even QDOS was 1980 at its earliest.

However, "The original FAT" was used by Microsoft's 'Stand Alone BASIC' and that was the late 1970s. FAT was originally written by Marc McDonald.

[Anonymous Coward]

And so we move on to stage two ..

We've had the Embrace phase, now we're moving to Extend.

Admittedly it's not quite as bold as how they butchered the Kerberos spec for their own use, but 30 years of behaviour suggests to be careful. As far as I can tell, there has been very little changing of spots on this animal underneath the extensive marketing camouflage.

[oiseau]

Re: And so we move on to stage two ..

... is the first stage in what Microsoft hopes will be its inclusion in the Linux kernel ...

I don't like this ...

Not at all.

30 years of behaviour suggests to be careful.

Suggests?

I think it actually defines anything M$ related.

This type of thing will end up rotting the Linux ecosystem from the inside.

And then it will be too late.

O.

[bombastic bob]

Re: And so we move on to stage two ..

it's really ok as-is with fuse support. FreeBSD has fuse support in the kernel. I can't recall if fuse support is build into Linux now [if so, excellent!]. So fuseFS is probably the best way to put ANY file system with patent or other encumberances into Linux, since it's not required to be "part of the kernel" [but might as WELL be]

[Steve Graham]

Re: And so we move on to stage two ..

Yes, Linux has FUSE support in the kernel. I use it to mount my Android phone via MTP (another Microsoft invention).

Recent versions of Android have dropped support for direct USB access to the filesystem (which was of the FAT type) and you have to use MTP, which is a file-based protocol (and hence slow and cumbersome). I think Android generally uses ext4 internally now.

[Anonymous Coward]

Re: but the alternative

is a bunch of sado masochists who worship an abusive master. At least with MS you know money is guiding them. God knows what makes grown ups complain when a graphics editor gets a name change to something that doesn't imply bdsm. You roam these boards repeating the same tiresome bullshit about "Embrace, Extend, Enjoy", hijacking every constructive MS conversation with half formed opinions about doing it wrong and claiming some kind of moral high ground because you use a different operating system.

Roll on stage 3.

[Updraft102]

Re: but the alternative

The existing name of GIMP doesn't imply BDSM, or ableism, or anything else offensive to anyone. It means GNU Image Manipulation Program, and that's all it means. God only knows what makes some alleged grown-ups choose to read into it meanings that are not there and were not intended, then choose to become offended by the meaning that was never there until they put it there.

[PerlyKing]

Re: The existing name of GIMP doesn't imply BDSM

You may be correct that it doesn't imply those things, but a lot of people sure as hell infer them.

The name was a cute joke for a bunch of nerds when it was a minor project, but if it's got to the stage where it's holding back more widespread adoption then "a rose by any other name would smell as sweet". Let's see how Glimpse does.

[Ken Hagan]

Re: The existing name of GIMP doesn't imply BDSM

Odd. Yesterday's article made it quite clear that people were objecting to "gimp" because it meant something to do with disability. Now you are stating that it is offensive because it means something to do with something other than disability.

When you grown-ups have finished your terribly mature and adult discussions about which particular alternative meaning is causing you offence, perhaps you could get back to us nerds with a specific complaint to which we can assign the attention it deserves.

[John Brown (no body)]

Re: The existing name of GIMP doesn't imply BDSM

"Odd. Yesterday's article made it quite clear that people were objecting to "gimp" because it meant something to do with disability. Now you are stating that it is offensive because it means something to do with something other than disability."

Yes, it is odd. I don't think I've heard gimp used as a reference to disability for at least 30 years. And even then it was rarely used in a derogatory way. Phrases along the lines of "I have a gimpy leg/he has a gimpy leg" was about the extent of the historic usage that I can recall.

These days, the word gimp, to me at least, immediately brings to mind the icon ------------------>

Either way, it's not a great choice of name for a great graphics editor.

[Anonymous Coward]

Microsoft

The comments here talk about Microsoft as if it's a single entity with some evil master plan.

It's not. It's over 100,000 very smart people who by and large are allowed to do whatever they like whenever they have a good idea. The simple truth is that this is happening now because someone finally put some thought into it and realised it's the right thing to do. If you want "MS" to do something, get involved in the community and find someone that works there who is interested in your idea, or get a job there and do it yourself. By contrast, if none of the 100,000 people think about something, they won't do that thing. No evil plots, no master plan

I mean sure, management try to influence the chaos now and then...with varying success...

[Anonymous Coward]

Re: Microsoft

Not true! It isn't!!! They have an evil underground lair in a volcano!!!

[Michael H.F. Wilkinson]

Re: Microsoft

and a white Persian cat, I assume?

[Anonymous Coward]

Re: Microsoft

Given current sentiments that would have to be an American cat now, but otherwise yet.

Also, it's not 100k people doing random things. They're directed by a much small group of people, easily identified because they typically do not have pesky problems like having to pay a mortgage or school fees for the kids, other than maybe having to pay some extra to get them through school even if they are as dumb and incompetent as their president.

It's this group that can be suspected to follow the same playbook they have been using for a few decades now, for the simplest of reasons: it worked, and it made them a fortune. Money is the strongest argument against change for any US company, Microsoft doubly so.

[Maventi]

This is only a good thing. Although overdue, better late than never. I really never thought I'd see it this soon.

It's clever too; my understanding of this is that it's only open source implementations that are protected from patents. It's likely that proprietary implementations (e.g. in cameras) still need to pay fees, so if anything this move may help push exFAT adoption and in turn wring more money out of those implementing exFAT but not wishing to join the open source bandwagon. Just a theory.

[Anonymous Coward]

Microsoft part 2

I wonder if MS has got a whiff that their are changes in the air for media hardware manufacturers. Fed-up with licence fees which MS dines on, Ext4 now look more attractive. Ext4 is more reliable and is licence free and would allow manufacturers to break the chains that inflate the prices of products not to mention Audio, Video and photographic equipment manufacturer would benefit. File transfers would improve with Nfs which is what Smb should have been. With more Wifi connectivity this would open a truly portable system. I'm opposed to doing deals with a company who will sue at the drop of a hat and changes it policies on a whim. In short “Beware the Jabberwock, my son! The jaws that bite, the claws that catch!"

[sabroni]

Re: allow manufacturers to break the chains that inflate the prices of products

I'm sure those prices will come tumbling down, but just in case, don't hold your breath.

[Brian Scott]

Re: Microsoft part 2

Ext4 on removable devices is just as stupid as NTFS (or UFS). They are both designed for permanently mounted storage on a single device. Both contain security features that are just plain simply wrong for transport between devices (for a removable device, the best indicator of permission to read or modify a file is physical access, with encryption for actual content security). There is little point in having UIDs or GIDs and associated permissions or ACLs when the corresponding mapping to real users or groups is a very individual system thing.

Furthermore, the standard for Ext4 is simply 'whatever is currently in the linux kernel' and in true Linux style is a movable feast (exactly the same is true for NTFS of course).

Neither filesystem takes very kindly to being physically ejected without being unmounted first, I would have thought a fairly basic practical requirement for any portable filesystem.

FAT used to be the right sort of answer in the past but has problems with modern media and file sizes.

What we really need is something purpose designed for transporting data between different systems on modern media. This is what ExFAT was designed for.

The problem has been that Microsoft's licensing ($$ for each use) has made it impossible to use in open source since there is no easy concept for number of use's even if some organisation was happy to pay the money. Not being viable in open source effectively means it is dead in this day and age so this is effectively microsoft trying to save it's life.

Now if only someone would fix the performance of ExFAT on Macos with files in the TB size range.

[rcxb]

Re: Microsoft part 2

There is little point in having UIDs or GIDs and associated permissions or ACLs when the corresponding mapping to real users or groups is a very individual system thing.

FAT's hidden and system attributes make no sense for removable media, but they're there.

Of course UDF is DESIGNED for removable storage, so it's absurd to claim it's features are an impediment to removable media. In fact those permission systems have very real, utilitarian purposes, even on read-only media, but it's not worth getting into the gory details here.

While individual user ownership isn't particularly helpful, the difference between normal user and root or Administrator writability is a very useful distinction that should be preserved between systems. Last time I really dug into it (which was quite a while), Windows had NO ABILITY to prevent other users on your system from reading/writing to the drive you inserted, if it uses a file-system like FAT32 which lacks permissions. So you're opening yourself up for sneaky data theft, worm infections, and silent data corruption.

Furthermore, the standard for Ext4 is simply 'whatever is currently in the linux kernel' and in true Linux style is a movable feast (exactly the same is true for NTFS of course).

You should take a look at /etc/mke2fs.conf on Linux. It controls which features of a file-system are turned on or off when you create a new one. It's very easy to remove a few flags from that file and create a file-system that will work perfectly with a very, very old system. It is quite properly versioned in it's way with those file-system flags ensuring you NEVER lose backwards compatibility. Even today you can create an EXT3 file-system that will be compatible with the very oldest, earliest implementations, and that's two-decades old, and had many features added through the years.

Neither filesystem takes very kindly to being physically ejected without being unmounted first, I would have thought a fairly basic practical requirement for any portable filesystem.

Quite the contrary. Modern file-systems like EXT4 handle the power suddenly being pulled far better than FAT32, which doesn't have a journal or other metadata to make it easy and reliable to repair.

The problem is the memory caching the OPERATING SYSTEM does to improve performance on ANY AND ALL file systems it knows about, and not some design of the file system. Any file-system can be set to flush to disk frequently and not keep large amounts of data cached in RAM. Changing that is just a check-box (or is it a drop-down menu?) on Windows, and a mount option on Linux... And indeed, Windows automatically uses different options for (removable) drives connected to USB, than it does for your internal drives.

[HobartTas]

Re: Microsoft part 2

Regarding "Neither filesystem takes very kindly to being physically ejected without being unmounted first, I would have thought a fairly basic practical requirement for any portable filesystem."

Perhaps instead we should use some open or pre-29 version of ZFS even though it probably is even worse at "doesn't take very kindly to being physically ejected without being unmounted first" but due to its superior features like check summing and time stamping for each and every block it could easily repair itself quite quickly when it's mounted and brought back online.

[antonyh]

exFAT, bleugh

Given my experience with exFat - formatting a device on Mac was unreadable on Windows, and vice versa - I avoid it at all costs. Not being able to read it on Linux was a bonus, a barrier stopping me from using this crap.

Add to that my Synology NAS wanting $3.99 for a driver (not an unreasonable amount) but needing me to create a login, add a payment card, etc, created another barrier for something that should be free anyway.

What the world needs is a free and open source, high performance, high reliability filesystem that can be used in embedded devices. It's not exFAT. Even the name makes me think of cooking by-products, disgusting.

[Anonymous Coward]

Re: exFAT, bleugh

The trouble is, that and "supported by Microsoft" are basically mutually exclusive, and given the OS used by most people's computer's, Microsoft can prove to be very persuasive.

[DuncanLarge]

UDF

UDF was supposed to be the answer, a universal, free, patent unemcumbered filesystem that was supposed to bridge all OS's while supporting the quirks of each such as POSIX file attributes for UNIX like systems.

Still used in many cases by each of the main OS's yet incompatible due to how the OS's format the UDF partition.

What a wasted oppertunity that was!

Luckily someone wrote a script that can format a UDF partition (i.e on a flash drive) to be fully supported by Mac OS, Windows and GNU/Linux. So I typically format my flash drives as UDF using this script and I get a filesystem that does everything needed by each OS I happen to use it in.

[entfe001]

Re: UDF

yet incompatible due to how the OS's format the UDF partition

More than how they format, the problem lies on where the filesystem is placed:

MacOS expects the filesystem fill the whole device, and gets confused if the UDF filesystem is inside a partition

Windows expects a partition table for all non-removable devices and a UDF filesystem inside it, so when it encounters such device without a partition table it doesn't know what to do with it

Linux doesn't care at all as long as you mount the proper device (full disk or partition)

The "compatibility trick" consists on creating a partition table with a single partition starting at sector 0, including the partition table itself, so Windows sees that partition and MacOS sees an UDF filesystem filling the whole device

Note that for what Windows sees as "removable devices" there are no such issues and a partition-less, whole-filled UDF filesystem works cross-platform

[Doug 3]

tracking every one

Microsoft is famous for tracking what their competition is doing. This sounds like yet another way for them to try and track what Linux distributions are doing.

Still waiting for them to apologize to Barnes & Noble and the many others they threatened because of fake patent claims.

[carl0s]

What about non Linux use?

Does this make any difference for non-Linux use? What if I build an embedded stm32 project or Arduino project, and the FAT implementation uses exFAT features. Am I going to be taken to court by Microsoft?

Isn't Linux supposed to be about free software? It's not very free if Microsoft will still sue people for using it (or rather, compatible implementations) outside of Linux.

[J.G.Harston]

Does nobody remember FAT12? :(

[TRT]

Charging for such a ubiquitous file format...

must surely be living off the exFAT of the land?

[razorfishsl]

As long as they give up the patents and don't use it to try and strangle Linux at a later date , it's not a problem....

[Quetzalcoatlus]

Yeah they published the spec, and it's one of the most unreadable specs I've ever seen in my life, not because ExFAT is that complicated, but because almost every subparagraph is a reference to another subparagraph, which may or may not include another reference to another subparagraph recursively. It's worse reading than legal codes. It's rather like trying to read files off an ExFAT filesystem with xxd and less, if you can do that, you can read the spec and vice versa I guess it's fitting.

Word needs macros like Excel. It might have them, buried in a ribbon somewhere, seen by none for decades. Classic MS design. What's worse, no start menu, or the ribbon? MS will find an answer, by experimenting on all their customers.