back to article Windows 8 to get self-healing 'Storage Spaces'

Microsoft will introduce in Windows 8 what it calls Storage Spaces – a method of putting drives into a virtual pool from which self-healing virtual disks can be created, with some resemblance to ZFS features. Details of these virtual disks – the aforementioned Storage Spaces – were described in a 4,400-word deep-dive blog post …

COMMENTS

This topic is closed for new posts.
  1. Sampler

    Sounds alot like how Windows Home Server works? I haven't tried WHS as I have a RAID5 array but when I was building it I did read up on it and it was a tempting idea for a home storage solution where I wouldn't lose any photo's/videos/work from drive failure.

    Being a bit of a nerd I went for the hardware solution but frankly I don't need the performance improvements over the software one - I guess many smaller home/business users wouldn't either.

  2. b166er

    This is good stuff from Microsoft.

    It will be excellent if you can assign individual files/folders their own level of protection and the subsystem just automatically deals with that.

  3. Anonymous Coward
    Anonymous Coward

    Wow! What an innovation!

    What an innovative and unique idea from Microsoft! Let me see if I understand it:

    You turn a disk drive into a, what do you call it, a physical volume

    You group those physical volumes into some form of logical volume

    You create virtual disks from those logical volumes, and those virtual disks can spread across different physical disks, and can survive a disk failing.

    What brilliant fellows Microsoft are! Wait - my Linux server just emailed me, something about a failure in LVM, a drive down, and falling back to the spare...

    1. Robin Szemeti

      Sounds like Drobo

      This "innovation" sounds exactly like Drobo, and no doubt a number of other NAS devices.

      I'm not really sure how it differs from LVM, RAID etc ... most of this has, as you say, been done before ... although I will bet good money that M$ has obtained patents for this "new" idea ...

      1. Boris the Cockroach Silver badge
        FAIL

        And then

        Sue every maker of LVM systems for infringing said 'patent'?

        Or just add it to the 238 patents linux infringes already but m$ wont say what they are

    2. Matt Bryant Silver badge
      Thumb Up

      RE: Wow! What an innovation!

      You can even have strict striping with LVM to make sure your mirror copies are on different physical volumes in the same logical volume. In fact, the only features I can see in the list that isn't in bog-standard Veritas LVM bundles are dedupe and thin provisioning, but that may be because Veritas (sorry, just can't bring myself to call them Symantec) have it in seperate products they charge extra for.

      If M$ do bundle it into the server OS without charging extra then it is a big deal, otherwise I'm not convinced it's a big step forward.

    3. Anonymous Coward
      Anonymous Coward

      Err...

      This is just an extension of the volume management services already offered by Windows. It already does RAID 0+1, 5, concatenated filesystems and the like and has done for years, it's just that each version of Windows has more functionality and better options. I'm not sure if I'm correct in remembering that the use of LVM in Linux has only really come in since 2008, ie: the last major revision of Windows server. However the use of logical volume managers in UNIX has been around for years, certainly since the 90s - did Linux copy Veritas foundation suite? No, it's just a natural development of required functionality for a modern operating system.

      1. Alex.Red
        Coat

        Hmm...

        The article below says that LVM for Linux was written in 1998 (10 years before you say it was):

        http://en.wikipedia.org/wiki/Logical_Volume_Manager_%28Linux%29

        1. Anonymous Coward
          Anonymous Coward

          re Linux lvm

          It may well have been around since 98, but it certainly hasn't been in standard use until really quite recently. I seem to remember it turning up around three or four years ago.

          I also note that the article says it was based on hpux's lvm...

          1. Michael Wojcik Silver badge

            re HP-UX LVM

            Yes, and HP-UX LVM is from circa 1992. AIX got there a few years earlier - AIX 3 had unified storage management with logical volumes on physical volume pools back in 1989.

            And of course other OSes for big iron (various mainframe and midrange systems) had logical volumes before that. The IBM System/38 (like its successor, the AS/400) completely virtualized all storage (the "single-level store" architecture). There was a filesystem with files, from the application perspective, but files had no particular relationship to disk storage allocation.

            That came out in 1979. So this sort of feature is at least 33 years old.

            Doesn't mean it's not nice to see it show up in Windows, though. As someone else noted, Windows storage allocation has gradually gotten better with each release, and this is the logical next step. (And third-party vendors had products to provide it already - now it'll just be built into the base OS.)

  4. Robin Szemeti
    Facepalm

    But the big question is ...

    Will the control panel app for this feature slide partially out of view a few seconds after you open it, and are just reading the info from the screen?

    No? Then why do you think its a good idea to have this "feature" on your website huh? Bored of your articles hopping about on the page when I am trying to read them.

    1. DAN*tastik
      Holmes

      The same happens on my computer too

      When it happens, I click on the down arrow key, or shift + space bar, half a second and I am back where I was before.

      If your operating system and / or browser turn this simple task into a disruptive one, you should consider changing either / both.

      Alternatively, I believe it is possible to open more than one article at a time in a different tab, by the time you have gone through the main page and opened all the new tabs you want , the page with the first article will be fully static.

      Another option would be to find all those who find the banners annoying and form a pressure group to make El Reg offer an ad free paid subscription version of their site

      In the meantime, we can just sit down for a minute and wonder if people tend to blow things up out of proportion, I guess

      Oh, and don't forget to click on the banners in the meantime.

    2. Matt Bryant Silver badge
      Thumb Up

      RE: But the big question is ...

      Taking a dig at the incredibly annoying M$ cloud ad at the top of all The Register pages, which makes the whole page jump up just as you start reading an article? Agree completely - make it static or take it off.

      1. Mr Anonymous

        blocked

        Too late, I activated a proxy on my gateway and blocked 2788 ad servers, inc ad.doubleclick.net, no more annoyances and no ad revenue for the reg from my office.

        Maybe the reg should think about their readers before accepting _disrupting_ ads.

    3. Anonymous Coward
      Anonymous Coward

      @Robin

      There are only 5 top stories. And you can see 5 icons which obviously represent said top stories (this is clearly seen by noticing the 'moving' highlight).

      As such; tried hoovering your mouse over said icons yet ?

      Unless we're talking about ads here (which I don't get to see anywhere on El Reg thanks to Adblock Plus) I don't see the big problem here. Its not as if those top stories hold a lot of text.

    4. steeplejack

      No problem for me with The Register, but then...

      NoScript and AdBlock Plus are present.

      1. kissingthecarpet

        I love those add-ons so much I'd go and live there

        FF + NS + ABP = surfing happiness.

        Its just a different web without the ads. My boss uses vanilla IE, & I look at his screen when he's viewing the same URL as me, & I almost can't believe its the same address. For those who don't know, not only does ABP remove the ads, it collapses what it can, as does NS(while also protecting me from exploits I don't even understand, let alone know how to avoid them all by myself). Faster loading, more info per square metre, no hideous flash ads, & the pages often look like there never were any ads. Lovely.

        1. Dr. Mouse

          El Reg is going to loose revenue.

          I have always disabled ABP etc on the Register to support them.

          These ads have become so disruptive I'm not going to bother any more. ABP back on.

          Take note: Readers are often more than happy to have ads, not so happy when they make a chunk of the article slide off their screen a few seconds into reading!

          1. Ru
            Paris Hilton

            "loose revenue"

            Is that like pocket change? Or is it the sort of revenue that has a bit of a reputation for enriching anyone who shows a bit of interest, if you know what I mean?

            1. Dr. Mouse

              Excuze ma spelin an grama

              s/loose/lose/

              I always mix them up...

            2. Michael Wojcik Silver badge

              "Loose revenue" is what you hope to pick up when, like ShelLuser up there, you are "hoovering your mouse".

  5. fLaMePrOoF
    Meh

    "Perhaps storage spaces are better carved out from pools made of many small drives than a few large drives."

    To my mind this is the only real advantage over hardware RAID; proliferation of data across larger numbers of more diverse drives could increase resilience, performance and build flexibility.

  6. Anonymous Coward
    Anonymous Coward

    Oh-oh

    Oracle will sue them!

  7. Mark2410
    Thumb Up

    Diy Drobo

    sounds to me like i can now build my own drobo with as many drives as i like then

    1. Arnie
      Thumb Up

      Try freenas. You can build array's quickly using whatever disks you have lying around. the only drawback is that you cannot turn a 4 disk array into a 5 disk one at a later date. You can however, replace existing disks in pool with larger ones. Not quite a drobo but considerably cheaper.

  8. Anonymous Coward
    Anonymous Coward

    Doh!

    These features have been in serveral Unix's since the early 1990's as well as Linux.

    So someone at MS has seen this and realises that this is just what they need as a USP for the next Windows release.

    These bozo's in Redmond really no need to get out a bit more.

  9. F. Svenson

    It's not ZFS...

    It's ADVFS from Tru64/DEC Unix.

    ...

  10. Fuzz

    Not the same as LVM

    To all the people saying this is like LVM, it isn't. Windows already has LVM type features, they're called dynamic disks and can be built across physical discs with mirroring or parity type fault tolerance.

    In this new feature (which isn't new as is pointed out in the article just new to windows) first you allocate a bunch of disks. Lets say you have 8 500GB disks. Then you create a storage space on your pool of disks and choose a fault tolerance level. Lets say that's a 3TB space with mirroring. Now in an LVM setup or a typical Raid array 8 500GB disks with mirroring (RAID 1+0) would yield a maximum partition size of 2TB so there's a difference. Now we start filling up the storage space, we add 1TB of data. One day a disk fails, note we have no hot spare, the system automatically shuffles the data around to rebuild the redundancy we've lost by using the spare space on each disk. As soon as this process is complete you're protected against failure again without replacing the failed drive.

    1. hplasm
      Meh

      It does sound like it will work, at least up to a point.

      But why do MS take something simple, like networking, and make it so oddly complicated? They've done it again with this.

      1. Anonymous Coward
        Anonymous Coward

        @Hplasm

        You say "Something simple like networking", then suggest that storage is simple.

        As a storage/SAN/backup person: I wish it were.

        1. hplasm
          Happy

          Surely storage IS simple?

          It's just retrieving the damn stuff in one piece that is the hard bit.

          Backups are Write Only, as any fule kno...

    2. Nya

      Re: Fuzz

      "new to Windows" actually it's not, it was in the first version of WHS as Sampler put in the first comment, which came out in 2007ish (my original WHS box is still ticking along nicely and always loved how it handled drives like this). Yes it wasn't included in version 2 (wasn't going to upgrade myself due to the loss of this feature) since they couldn't get it to work at the time, but it does seem like they have finally fixed it and tweaked it a touch.

      So all the whiners saying it's nicked off this that and the other, yes there are other things on the market which do things in a similar nature. But MS has had this for a long time, just not done a great deal with it till now.

    3. Anonymous Coward
      Anonymous Coward

      Sounds just like levelling on a SAN to me.. Providing you have enough spindles you should, in theory, never lose data due to a disk failure.

      1. Kebabbert

        Loose data?

        "...Providing you have enough spindles you should, in theory, never lose data due to a disk failure..."

        The problem is that in practice, your data is slowly corrupted, without you ever knowing it. Slowly, your data rots away, due to bit rot. So you are wrong.

        Have you heard about ECC RAM? Why do you use it? Well, because your bits in ram flips randomly, they "rot". Suddenly, a few bits flip on random. Therefore you need ECC memory sticks to detect the flipped bits, so the computer can correct the faulty bits.

        Exactly the same problem occurs on disks. The bits flip on random, but there are no ECC functionality to detect nor correct the faulty bits on disk. This is called silent corruption and a big problem when you store large amounts of data, over 10TB, say. Having raid and enough spindles does not protect against bit rot, as hardware raid, ntfs, ext, jfs, xfs, etc are not designed to look for these kind of errors.

        There are lot of research on this Silent corruption issue in this link. (initial research shows that zfs solves all these problems):

        http://en.wikipedia.org/wiki/Zfs#Data_Integrity

        1. Ru
          Headmaster

          "Loose data"

          Is that the sort of data that spills off the end of a buffer overflow? Or the sort of data that has a bit of a reputation...

          etc etc.

          What is it with all the loosers on here today?

        2. BristolBachelor Gold badge

          Maybe I am a bit late posting to this, but wouldn't bit-rot cause a read error, as the sector CRC would be wrong? OK so that means you have to do a read-check every now and then, but that is certainly something that my lowly NAS does which is just based on Linux.

          I saw some mention of this before and from what I could gather, the only difference with zfs is that it does a consistancy check using the data and parity. This means that zfs would pick up bit rot that had somehow managed to rot the bits but still have a valid CRC. (OT but in the end I didn't go with zfs because of the inflexibility re-sizing the array)

  11. Anonymous Coward
    Anonymous Coward

    Storage Spaces is somewhat like ZFS, although it [ ... ] lacks [ ... ] ZFS features.

    So, not like ZFS then. Why not just call it software RAID?

  12. Ken Hagan Gold badge

    In fairness to Microsoft...

    ...this is, at least, a genuine OS-level feature rather than yet another bouncy graphic for the vacuously inclined. It is, in fact, the first I've heard that Win8 will be different (at an OS level) from Win7 and therefore conceivably (*) worth shelling out money for.

    (* Probably not, in practice, since I'm quite happy with my existing backup strategy and don't require five-nines uptime, but I dare say it will appeal to some.)

  13. a_mu

    software raid, windows NT rules

    oh dear,

    why cant redmond just use existing stuff,

    raid , zfs , vlm.

    1. Anonymous Coward
      Anonymous Coward

      @a_mu

      Oh dear, why can't commentators understand what the article is about, before making a knee jerk comment about MS, when they clearly know nothing about Windows.

  14. Anonymous Coward
    Anonymous Coward

    I thought MS were putting this "any disk into a fault tolerant disk thingy" in the *second home server version, but admitted they couldn't get it to work and dropped it, thereby upsetting beta users. Maybe they took on some _more_ **Unix guys to fix it for them.

    * I'm sure this was in an old reg article.

    ** Unix guys fixed a pile of stuff in NT.

  15. Daniel B.

    Meh

    I'm guessing that all of this is moot, as home users will probably get shafted. What do I mean? Well, MS is bound to put this feature as a "Business/Ultimate only" or whatever, so everyone buying PCs will have to pay double to MS (the OEM license, then the upgrade) to get Storage Spaces. Already the case with their software RAID, Remote Desktop, Hyper-V integration stuff... Bad bad MS.

    My company is actually planning on going Apple thanks to that. They rather pay less on MS licensing. :/

    1. Anonymous Coward
      Anonymous Coward

      Maybe, but they sure shown differently when looking at Windows 7. I get the feeling you're overestimating the needs / requirements of the average Windows end-user. This is not to say all end-users are stupid, but their demands are hardly as high as you portrait them to be.

      Just look at the difference between Windows 7 home-premium and professional. The main difference is networking. Professional allows access to several networks (domains, vpn, etc.), provides more network access to some applications (backup for example) and finally provides options for using native XP software through means of "XP mode".

      Which doesn't even mean that home-premium users can't use MS Virtual PC with a virtual XP environment. Its simply not supported "out of the box" so to say.

  16. Anonymous Coward
    Anonymous Coward

    oh for hevens sake.....

    yadda yadda yadda.... linux already does this along with various other bits of kit... so what...

    the point is, that up to now Windows does not, so they are adding it to the feature list.

    Will it make windows better? yes... is it a worthwhile addition? yes... does Microsoft claim this is a brand new technology that they have invented? no,

    Give them a break ffs...

  17. Anonymous Coward
    Anonymous Coward

    When do we get end-user innovations?

    Now, while all the latest news around Win8 isn't all too bad, also not too exciting, I have to wonder...

    When do we get to see new developments which really make the lives of the end-user easier? So far all we got were more technical aspects; mounting (accessing) ISO files, a new recovery method and finally this "drive-pooling" feature.

    I've grown a liking towards Windows 7 last year and learned quite a bit about it. Enough to think that there is a (slight?) chance that people disliking Metro might be able to work around it.

    But, my point: while all these latest developments are nice for techies they'll mean nothing for end-users. Some of my customers (end users) have worked with Vista for years now and still didn't know about the "previous version" option. So when a file was deleted by accident (and not to be found in the waste bin anymore) they feared it was gone for good and called me.

    Being able to access ISO files? Its an interesting feature, but its too late. Most people will already use virtual disks (I like MagicISO) or a mere archiver like WinRAR if all they need is to get a single file.

    So my prediction is that it may very well turn out that the biggest innovation / change in Windows 8 will be Metro (from an end-user point of view). And quite frankly I don't think that's going to sell very good; I still foresee a new "Vista disaster", but maybe even larger.

    1. Ken Hagan Gold badge

      Re: end-user innovations

      It's an operating system. It's job is to be boring.

      If you want end-user excitement, buy some applications.

    2. Ken Hagan Gold badge

      Re: previous versions

      Of course, the previous version is still there for *you* to find precisely because the end-user didn't know about the feature and therefore hadn't either switched it off or recently purged all previous versions. Making the feature *more* visible to typical end-users might actually make it *more* likely that they lose data.

      VMS had a similar feature and a sub-population of end-users who relied on the fact that a document had multiple personalities and so you could store important data in different versions. It was great until they exceed their disc quotas and their sysadmins "fixed" the problem for them by purging all older versions of all their documents.

      There's no substitute for a decent backup regime. I know it's boring, but if you can't calm down enough to put one in place then I'm afraid you are just too disorganised to use a computer.

      1. Anonymous Coward
        Anonymous Coward

        @Ken Hagen

        This is why you never allow end users the ability to manage their own data or backups. Let them load/save/delete/copy their own stuff, but no more. At a previous company I worked for, we found that the end users were using NetApp snapshots to store data in such a way that they got a larger quota because of the multiple copies. Realising the implications (ie: loss of data, etc. etc.) we took away the ability for the end users to have any access to their snapshots, waited for them to call the helpdesk and returned their missing files along with a brief lecture about file management and why the quotas were there in the first place.

  18. Chris Ridley
    Happy

    USP

    Ermmm.....Will this be available as a clip on pack to the back of you nice shiny new Win 8 tablet?

  19. Chris McFaul
    Stop

    parity spaces

    According to the FAQ there is no such thing as dual parity support, single parity is the only option , contrary to your article

  20. Levente Szileszky

    If you keep your Windows desktop running 24/7 then...

    ... it's OK, they just added a feature others already have.

    But if you like me, prefer to use your nice, fast machine to _do_ things, perform tasks etc and keep your inactive files on a small RAID5-RAID6 NAS box that's big and can handle all sorts of other things 24/7 while running noiseless, tucked away in your house, well, then it's not for you, I mean not for us. :)

    1. hplasm
      Boffin

      If you keep your Windows desktop running 24/7 then...

      it will stop working.

      1. Marty
        Stop

        fud and rot !!!

        "If you keep your Windows desktop running 24/7 then...

        it will stop working."

        what a load of rubbish.....

        well these days it is, in times gone by it may have been a issue but these days you can leave a windows 7 machine running non stop for weeks without a reboot...

        the proof.....

        well, my missus and her laptop....

        she never shuts it down or reboots it, the lid gets closed when she has finished what she was doing and puts the computer into standby or hibernate... which is near enough the same thing as leaving it switched on, it just reloads its last state back into memory...

        she can go for weeks and weeks without shutting down and rebooting, in fact, if it wasn't for the fact that some updates require a system restart, it probably never would get restarted

  21. jake Silver badge

    Whatever.

    Microsoft has been irrelevant here at Chez jake for over two years.

    As for the advertising folks are complaining about ... see my first paragraph.

  22. steeplejack

    If it's new Microsoft software, beware of bugs!

    I'd be very wary about entrusting my data to any newly-coded filesystem. Its in Beta now, but give it PLENTY of time to get debugged (preferably years, bearing in mind that users lost data with Microsoft Home Server changes).

    1. Anonymous Coward
      Anonymous Coward

      Really?

      I'd give MS just as much time as anyone else before I use their software... A month or so after the initial release, for personal use, maybe SP1 or the first point release for business use. This rule goes for UNIX, Linux, Windows, commercial software, FOSS, new Array firmware/features etc. etc.

      NTFS and its underlying volume management is a pretty mature product and I can't remember the last time that I heard of anyone losing data because of the filesystem/VM. Come to that I've been a storage person for some 15 years and I can't actually remember having heard of anyone lose data due to NTFS/NT Volume manager.

      1. Anonymous Coward
        Anonymous Coward

        Lost data indeed

        Just google if you don't know. Here is the first link I found

        http://blog.jamesbayley.com/2011/03/14/windows-home-server-reinstall-disaster-use-raid-next-time/

        NTFS might be mature, but it is definitely not the cat's meow. Compression possible only for small (4k max ) clusters, 64k max cluster size, 256TB max volume size, 16 TB max file size, etc.

        Nothing to brag about.

        Also: ZFS rules. :)

        1. Anonymous Coward
          Anonymous Coward

          Err?

          Of course you're going to find problems with any filesystem, if you search for it on Google...

          As for the supposed limitations, they sound fine to me - Most, if not all, major disk arrays use 64k track size, compression isn't really a required technology any more and it's still there, if you must have it. In 14 years working for major UK financial companies, mainly in storage, I have yet to see a single file on any system (UNIX, Linux, Windows etc) even close to 16TB and multiple file systems can be used in the unlikely event that you need more than quarter of a petabyte on a single FS.

          1. Anonymous Coward
            Anonymous Coward

            Because the "supposed limitations" (they are real you know, take your blinders off) sound fine to you doesn't mean they are fine.

            I bought a really fast HP DAS array of 25 15k disks in RAID10, configured at NTFS's limits with 64k clusters and no compression for my data warehouse (and io-fusion solid state storage for the temp area).

            The hardware is really fast, but NTFS sucks here, and it'd probably be 30-50% faster with 256k clusters and compression. Unfortunately there is a NTFS-only policy, which means we'll spend more on a second DAS just to increase the speed. :(

  23. kissingthecarpet
    Linux

    I think its a bit silly

    that windows doesn't let you use other filesystems apart from NTFS & the FATs, i.e windows doesn't even seem to recognise other filesystems partitions as actual filesystems, let alone reading the data AFAIK. This may have improved recently. If it has, it hasn't got much publicity.

    PS WTF happened to WinFS then eh?

    1. Anonymous Coward
      Anonymous Coward

      err...

      It does, you just have to get a driver. I believe there is an ext? driver, there was also briefly a vxfs driver.

      It's worth pointing that most Unix derived file systems don't support all the features that windows really needs such as acls, alternate datastreams etc.

      1. kissingthecarpet

        Yes, but

        Its not like windows says "there's a partition but I can't read it", it just ignores them or regards them as corruption. You'll ne'er see a message suggesting you "get a driver". Where from? Windows won't find it for you that's for sure.

  24. Charles Manning

    Smoke and mirrors space more like

    Considering MS's record at delivering file systems, don't count on anything until it is actually released in a final release.

    WinFS (and it's precursor OFS) was promised in every release since 1990 but never made the cut for delivery.

    The usage model looks far too complicated and confusing for Joe Punter. The people just want something very simple that just works. Apple Time Machine looks pretty neat.

  25. Anonymous Coward
    Anonymous Coward

    Self healing?

    As in hot-spare disk in case of failure, which is what I understood?

    Or as in no parity write-holes, and sum-checked data that can be scrubbed for error?

    If the former, nothing new.

    If the latter, ZFS is being emuled in some of its most interesting characteristics (and more if disks can be added to a vdev).

    Which one is it? Thx

  26. Slabfondler
    Trollface

    You're all misin gthe big, er, um only picture!

    Cleary the news here is that from two disk of 2 TB capacity you can create a 10 TB filesystem which is mirrored and striped and loose and all that...that's pretty darned awesome how they can make all 10 TB of that fit on those two disks that only have 4 TB of capacity.

  27. Anonymous Coward
    Anonymous Coward

    isn't this Rabin's Information Dispersal Algorithm?

    Mention of a "quorum" and "not RAID" seems to suggest it is. I must say I love the IDA for providing data resilience--to the extent that I've written a free set of libraries for implementing it (*), and have used it to back up ~1Tb of my own data by spreading shares over 4 USB drives (with a quorum of 3, so that if any one drive fails my backup is still viable). I've tested it enough to be confident that recovery will work if a drive does fail.

    It's nice to see that other people see the benefits of using this scheme, though I'd note that it's not a drop-in replacement for a RAID system--you need at least 3 disks (preferably more) before you see any advantages over plain RAID, and the article also mentions that the IDA calculations are done by the CPU since no controller can (currently) implement it. Though I guess that the target system (lots of small disks, beefy multi-core processor) can make it work very well.

    (*) See gnetraid project on sourceforge, or download Math::FastGF2 and Crypt::IDA modules from CPAN.

This topic is closed for new posts.

Other stories you might like