Backup your files with CrashPlan! Except this file type. No, not that one either. Try again... CrashPlan has banned a bunch of file formats from its online backup system aimed at small businesses. The company (tagline: Automatic Data Loss Protection for Your Small Business) sent out emails on Friday with the new list of forbidden formats. For users of CrashPlan for Small Business, these include application directories …
So it's the Plan to Crash my data ? No thanks. I have a backup system, thank you, and I do need to backup parts of the stupid Program Files directory which everyone and their dog seem to use as storage locker for data their code dumps there.
Who are you to decide not to accept data from a given folder anyway ? Either you backup the data I want backed up, or you can take a hike.
I used to use CrashPlan, but I won't touch proprietary backup systems ever again.
The replacement I now use is Duplicacy - which allows you to run your own servers, backs up to a wide variety of cloudy services, does cross-machine de-duplication, handles encrypted backups, and is open source. Development supported by donations and paid-for Windows GUI. Web GUI in development.
Agree. I've encountered the file type exclusion problem with too many backup tools. I now encrypt first then upload, not just for security, but to prevent the backup provider applying some stupid rules to they think is should or should not be backing up.
I had a similar thing at work where the IT team decided apply a new rule excluding everything outside of My Documents. What a move! They didn't tell anyone. I fortunately noticed before a restore was needed.
Fair point. It'll probably happen eventually.
IIRC it's Windows file search, the most hopeless search tool ever, that refuses to touch files it doesn't recognise. I guess it's so it can speed up searching of things like an mp3 file where it'll just search the artist/title/album/etc tags rather than doing text search within the music. A file with no extension it just skips over. So there is precedent for this sort of thing.
Encrypting, changing names, moving directories etc is still pissing around trying to get round a policy which is user unfriendly at best, all it's doing is deferring the inevitable; that they'll update the policy to catch you out anyway (e.g. excluding files of unknown types).
To be honest, it takes seconds to run the "file" command against any file and it'll tell you what the real type was, whether the extension is correct or not. It'd certainly spot VM images.
Of course, all you need do is what you SHOULD BE DOING ANYWAY. Encrypting anything you "backup" to random third-party servers somewhere else in the world.
Proper encryption, resulting in a ".bak" or similar file, rather than just copy/paste your VMs as they are for anyone who works at that company to poke through.
> Encrypting anything you "backup" to random third-party servers somewhere else in the world.
And it's not like these are Joe User files. This is apparently designed for "business critical" info.
eg stuff that in the wrong hands could put the business in jeopardy. Or at least seriously knock the price for buying it down.
>Encrypting anything you "backup" to random third-party servers somewhere else in the world.
The backing up isn't the problem - just enable inbound SAMBA, FTP et al connections to access to your systems from the Internet, the only problem is maintaining a reliable list of "random third-parties" to which you can make restore requests...
At a defence company I used to work, our company joke was that we relied on the Chinese Embassy for off-site backup. (Obviously there was a proper backup of everything - except email!)
Until one day a random engineer was asked the backup question by an AS9100 inspector, and "the Chinese Embassy" was definitely not the correct answer the inspector was expecting. Que a company wide meeting/rollocking where the actual process was explained in infinite detail :-(
Bareos is probably the one new users should be pointed at, being a fully OSS fork (from a few years ago) with a large user base and active developer Community.
They forked when Bacula started pulling dumb stuff, trying to convert everyone to paying members.
Repeat after me:
"The cloud" is just Someone else's computer
Now, bear in mind that if your computer has a problem (crashes, hardware failure, network loss, etc. etc.), the power is in your hands to do something about it (even if that something is scrap it, replace it, and restore everything from backup). If "the cloud" has a problem, the number of things you can do about it is zero (unless you count being put on hold by a call centre drone).
So, the logical thing here is that if you need to back stuff up reliably, the only sure-fire way is to do it yourself. If you're particularly paranoid, you ship those backups to another building, in case the building your computer is in burns down. Depending on the frequency you want to do this, and the amount of stuff you need to back up, the actual solutions may vary from copying a file to a pen drive, to live log-shipping and replication over multiply redundant secure network links to multiple secured off-site bunkers.
'"The cloud" is just Someone else's computer'
For me, the cloud is offsite storage of critical data. It is encrypted with a key the cloud provider does not have and they don't get to choose which file types. Whole system backups are on-site but on NAS.
Most disasters can be recovered using the NAS. In the case of something far more serious (fire etc.) if no local backups are usable, at least critical data will still be accessible (I check this from time to time.)
This arrangement is as near as I can get to an ideal situation without spending a fortune, until I can put the NAS in a separate building (something more secure than a shed!)
My data is held locally, protected by RAID and was backed up to Crashplan, which until now had been flawless. I also periodically ran an LTO (yes, that old tape) backup.
Most of my data was also only on the server when it was shunted off the original device, so I had 3 copies for a point, then typically two.
The point being that whilst I know a Cloud provider may be ephemeral and your data may go poof, the management of this implementation leaves a "lot" to be desired. Like "management", decent comms etc - I can't believe they just looked at the potential disk size reduction and went "What could go wrong?"....
"My data is held locally, protected by RAID and was backed up to Crashplan, which until now had been flawless. I also periodically ran an LTO (yes, that old tape) backup."
Perhaps I should have spelt it out better, although I assumed everyone would understand the implication that the three copies were backup copies.
The live version is not "a copy". Its the live version of the data you are seeking to protect. And thus minimum three copies of that data.
"I thought there was always this IT motto that "for anything important, keep minimum three independent copies, minimum one off-site"."
It's been a few years since I've used CrashPlan in a different role, but we used it for end user laptops.
In theory they were supposed to backup everything important to dedicated file shares that were then backed up as part of the standard company backup policy (disk-to-disk-to-tape).
Only many of the users that we were trying to protect didn't come into the office due to travel or working at customer sites, would forget to VPN in/manually start backups from home and if there were in the office, it was only for meetings, so CrashPlan was a life saver if laptops were lost/stolen/destroyed.
Horses for courses...
I thought there was always this IT motto that "for anything important, keep minimum three independent copies, minimum one off-site".So, I'm afraid, I feel no sympathy whatsoever with people who put all their eggs in one cloudy basket.
So people using a *paid* for service aren't allowed to complain when files of theirs are deleted?
Besides, they could even be following the rule you've quoted, but have just effectively lost their off-site backup.
"Besides, they could even be following the rule you've quoted, but have just effectively lost their off-site backup."
I would suggest to you that if they were following the rule I quoted and lost one copy, those people would not be going around like a bunch of whinging babies crying from the rooftops on twatter.
Because if it were the case they only lost one copy, then they could:
(a) Re-create that copy from the live data; or
(b) Re-create that copy from one of the two remaining backups
For this you need at least one of the backups systems to be working properly. It is absolutely right that people should complain about a backup tool that stops backing up data - especially for data where the system has been tested and proven to work.
You can have a dozen backup systems in parallel but if you tolerate crap like this from all of them you may still not have a backup. You want multiple GOOD backup systems so in the extremely unlikely event one of them fails there's another that almost certainly hasn't. And you should still be annoyed by the one that failed.
Those "whinging babies" may well have done all that and still be annoyed (because they're the kind of people who look after their data) that the people they pay for a backup service decided to delete backups without warning. They are also serving a useful purpose by doing (c) Warning others about this provider.
Part of the problem here is how long it takes to establish that offline copy. I used Crashplan for a long time, and it wasn't quick at uploading. I had multiple TB uploaded from domestic broadband. I'm not even sure it completely finished. My offline copy is now a bunch of HDD in a storage locker, and even then it doesn't cover everything because it's added to infrequently as well as running the risk of bitrot.
In 2019, backup is not a solved problem for non-enterprise users.
If anyone was to act to blasé about it, then why are they bothering using that companies services in the first place?
Remember, this isn't:
"oops, one of my backup providers has failed... Just as well I have another 2"
But more:
"One of my backup providers has intentionally deleted my files. Not only did I pay for this service, I now effectively have one less backup provider than my carefully worked out backup strategy recommends"
I would suggest to you that if they were following the rule I quoted and lost one copy, those people would not be going around like a bunch of whinging babies crying from the rooftops on twatter.
There's a certain inconvenience to uploading multiple TBs of data to a new provider. It is likely this 'new requirement' to start over again that pisses people off. That and the moving of the goal posts.
They're not the cheapest, but they've got ZFS snapshots, rsync / sftp access, and have been quite brilliant for keeping my important files and servers backed up, and they've got technical tech support. Also offer a nice little rsync app for Windows for backups. It's no-frills and no bullshit, with geo-redundancy if you want it. I use it to back up our on-site backups to off-site, and it's remarkably easy to retrieve backups from a month ago if you need them. Clearly aimed at technical users, which I suspect would suit most of the readers here...
rsync.net FAQ if you want a nosey.
Just storage. If you set up snapshots, with extra for snapshots if needed. As it's ZFS based, it only occupies as much extra space as an incremental backup would from the earliest snapshot on the account. Can set up custom snapshots too, I go dailies plus 4 weeklies.
+1
I've been using tarsnap for years. https://www.tarsnap.com/bounty-winners.html :-)
They have mac and windows (cygwin & subsytem-for-linux) support too...
Used them for years to backup my PC, I had a lot of family stuff going back decades which I'd spent weeks scanning in, mostly old photos but tons of records etc. Was a great setup and I had no issues with the service.
Then they said to bugger off, didn't want my money anymore as I wasn't a business.
Such a weird tactic.. I'd never so much as contacted support in 4+ years. They seem to have adopted IBM's mentality when it comes to business, go big or go bust!
Probably be the later in a few years.
has just jumped ahead of Office 365 for the "I wish we hadn't called it that" award for Most Ironically Named Business Cloud Service 2019.
What the hell is the point of a cloud based backup service that doesn't allow VM disk images? It's VMs that crash, not Word documents (erm, hang on, sometimes it is Word documents.....)
Putting the infrastructure in place to filter files sounds like it'd be more time/effort/cost than just sucking it up and buying more disks and bandwidth. Especially since, like all backup services, they charge per gigabyte / TB. Mass storage, especially of the performance level you'd expect for backups, is ridiculously cheap nowadays, you can get disks at less than 3 cents per gigabyte and you can get a 90-bay JBOD chassis for about $10-15k. Bandwidth is fairly cheap too.
> Especially since, like all backup services, they charge per gigabyte / TB
Looks like Crashplan is unlimited. So they probably have a small number of users backing up enormous amounts of data. What Crashplan *should* do is stop the unlimited stuff and start a pay-for-what-you-use system, that would encourage users to be selective about what they backup... and if they don't it's making more money. Instead it looks like they're trying to keep the unlimited aspect while sneakily limiting what people store which can only end in tears for the users.
I learned, long, long ago, that if I don't have the data in my grubby little hands, usually now in the form of a USB flash drive, or DVD, (or, decades ago, on a tape that I physically had possession of, or on floppy disks), then the data didn't really exist. I've heard, way too many times in my career, "Oh, that tape doesn't exist.", or "We erased/scratched that tape.", or, "We lost/broke that DVD.", or "We don't back-up those file types.".
Obviously you need a copy on your own hardware, but what if you don’t have an offsite location of your own? How do you protect against fire? You want to be able to buy new kit, restore your virtual machines and data from a cloud backup and be back up and running in a few days.
If you can't verify that the offsite copy exists and is intact, then don't count on it being there.
The backup companies revenue comes from promising "we'll keep you data safe". Storing data, internet bandwidth and competently managing your data to keep it safe all cost a great deal. The best business model for them is to discard all your backup data and apologise if you ever need to restore it and you can't. They could even fake an admin console that shows all your backup files so you feel secure.
Vicar: But my car was hit by a lorry while standing in the garage and you refuse to pay my claim.
Devious: Oh well, Reverend Morrison, in your policy... in your policy... [pulls out crumped piece of paper].... here we are. It states quite clearly that no claim you make will be paid.
Vicar: Oh dear.
Devious: You see, you unfortunately plumped for our 'Neverpay' policy, which, you know, if you never claim is very worthwhile, but you had to claim, and, well, there it is.
Vicar: Oh dear, oh dear.
Devious: Still, never mind - could be worse. How's the nude lady?
Vicar: Oh, she's fine.
I used Crashplan for years and was very happy with the policy of unlimited data without filters. Then they started getting sniffy about what I could and couldn't back up and put up prices. I moved to BackBlaze. They don’t back up all files either, however they are a bit more sensible about it and just exclude system and application directories. If you really want to back up that data you can always archive them to .zip and put it somewhere else on your disk. BackBlaze online backup is just £5 per month.
The also do 10GB of free cloud storage. You can link this with your backups. Backups are normally kept for rolling 30 days, so if you want to protect old versions of files you can put them into cold storage and it doesn’t cost you a penny. And if you have loads of data and want more cold storage the monthly costs and retrieval fees are cheaper than other cloud services.
Even if you don’t want or need BackBlaze the availability of 10GB of free public/private online storage is worthwhile. If you have good FTP software that might be all you need, and backup for free.
@microchip : Thank you for your kind words - we really appreciate your business and were pleasantly surprised to see rsync.net mentioned in this register article.
We have a very boring product - we give you a live UNIX filesystem (with your own .ssh directory) to do whatever you want with. You get configurable ZFS snapshots that you can browse right into and see your files as they existed X days/weeks/months ago.
If you're not sure what that means, our product is probably not for you.
I like boring. I have had far too much excitement at work over the last couple of months.
My own backup system at work is very similar. Rsync to internal server, and then an external server rsyncs the really important stuff to itself.
I don't have snapshots though. Probably could add them, but it currently isn't a business requirement.
"We have a very boring product - we give you a live UNIX filesystem"
I love boring. The more, the better. Screams and frights belong to a roller coaster, on an amusement park.
I want my data safe, and ad bored ass possible.
Just like my refrigerator: 15 years, not a glitch, and I don't even remmember the brand.
Boring is good.
Heh, didn't expect to see you jumping on here, it's a personal endorsement of "I use this, it's reliable and does the job well". You've been pretty reliable, and when I've had a problem, you've jumped on it and got it sorted. And you don't bother with fluffy support either, you answer honestly and get stuff done. I don't want something with pretty GUIs, I want stuff to quietly run in the background and not need me to worry about it.
Like people above said - generally boring and reliable. As a backup should be!
No, there are no charges or accounting for usage/transfer/bandwidth - you simply pay, by the GB, for the space you reserve.
We can increase the size of your filesystem at any time, with no downtime
involved - so there is no need to overbuy in anticipation of future usage.
Further, you are granted a +10% grace at all times, so you always have
some room to grow.
Finally, the system automatically emails you as you get close to your
limit, so there is never an unexpected filling of space.
Re: if you're not sure what that means, our product is probably not for you.
I LOVE that attitude. It saves on support costs being factored in for people who need handholding. Those who don't know what it means, but are sufficiently motivated, can or will learn.
I've used CrashPlan for about 10 years as a paying customer, though I hosted data for several friends until they removed that option. I've recommended it to colleagues and to several clients. I recommend them because they are "safe" not for the price or flashy features. Backups are just managing risk.
In that time, they've had many changes, but they've announced them with enough time for customers to plan and possibly migrate their backup methods. When I've had a problem, they've not only solved my problem, but sometimes even comped a little paid time as a goodwill gesture (when it wasn't their fault). When they dropped the consumer product, they offered a generous introductory rate — and data migration — to customers who carried over to their Business product. Even if they were to shut down the service, all my experiences with them give me confidence that I could, at the very least, order a drive with all my data before the door closes. #notlikemozy
This change seems poorly considered, badly executed, and involves a gross failure to communicate. It's basically everything that I do not expect from Code42. I earnestly hope this was a mis-step which they soon correct, and that it doesn't indicate a new direction for what has been a really top-notch company that provides a trustworthy product.
"But that seems pretty generous compared to how many consumer cloud services treat their punters – we're looking at you Flickr..."
Well, er, no, not really? Flickr didn't do anything to *punters*, i.e. people who actually pay it money. I have a Flickr Pro account and nothing at all changed for that. Flickr cut down what it gives away *for free*. That seems a lot less bad than screwing over paying customers, which is what this CrashPlan lot seem to have done.
I made a deal with a coworker who works in an office on the other side of the ocean. So now we just buy identical servers, and use ssh tunnels (via a pair of OpenWRT-based routers to connect our sites, and another ssh tunnel on the machines) to make them accessible to the other person. Then we just rsync our machines to directories on those servers.
Right now we both have a small ITX-based machine with 4x 10 TB HDDs in them. The systems run off a pair of 8 GB thumb drives running FreeNAS with ZFS on them for encryption and reliability. To keep things secure, we have IPMI turned on so we can remote into our system and enter the passphrase to decrypt the drive (that way, just because we are in physical possession of the machine, we can't decrypt any data on it). We also have the fact that we both have the same reason to trust in the other and stand to lose the same amount for breaking the trust.
We've been doing this for like 8 years now, every so often we'll go in together to buy new machines, UPSes and routers. Sure, it costs us an average of like $500 a year, but worth it to have 30 TBs of storage all to ourselves and no worries over whether we're being spied on, the service is just going to fall over in a way that is out of our hands, or suddenly find out that the company is deleting a bunch of files without warning...
I been using them for a few years and am pretty happy with them. I don't know about some of the claims that they lost their data without notice. I get a daily status report and warnings if a drive han't been backed up for 48 hours. I think the time period is use selectable. As for restoring files I recently restored a version of a file from a few days ago as daily versions are kept.
While it won't backup VM files directly it will backup mapped drives which works ok for me.
The backup service doesn't always seem to start with windows & needs to be manually started from time to time. As an inexpensive small business option, and my the only backup, it's ok for me.
Crashplan has always filtered on file extensions like this. I've been using it for over a decade. This isn't anything new? Maybe they hid it better in the past, but now the true nature comes out on some level of setup.
One thing they do well is continuous backup, changed files get backed up quickly, rather than on a snapshot schedule like most everybody else.
OOTH, its hard to believe they'll be in business with how they are turning away their core users. They want to sell to businesses, but won't support servers?
They want to turn into a security monitor DLP solution, but I don't know of anybody that wants a DLP unless its a compliance box to check.
"Crashplan has always filtered on file extensions like this. I've been using it for over a decade. This isn't anything new? Maybe they hid it better in the past, but now the true nature comes out on some level of setup."
They added more file extensions and whole directory trees to their exclusion list. I refer to the email they sent, because their support site is a big 404. We could argue whether software should not store config files and logs here, but many do.
/Users/<username>/Applications, /Applications/, /Program Files/, /Program Files (x86)/
"One thing they do well is continuous backup, changed files get backed up quickly, rather than on a snapshot schedule like most everybody else."
Completely agree. I love their service and use it as a benchmark for automated backup systems.
I've seen three..., no four backup solutions in my life which did not fuck up terribly sooner or later. Tar and dump being reliable but useless nowadays (unless you've got lots of cash for a serious tape library), backup2l and Cobian backup which have never ever let me down in ten years. The latter of course has lost a lot of useability since Windows stinking 10 has pretty much ruined incremental backups since your base partitions content will change unpredictabily and Windows still has no useful concept of separating content and executables.
>upload from Program Files, where a lot of applications store important data
Not defending the aptly-named CrashPlan in the least. But moving to Linux or OSX has the delightful effect of never commingling your data and executables again. Or indeed your executables and their configuration. And let’s not even bring in the supreme configuration isolation clusterf**k that is the Registry. Backup and security become much enhanced.
~, sweet ~.
"CrashPlan for Small Business was designed to back up and restore business files (e.g. documents, spreadsheets, presentations etc.)."
Now making it policy is like shutting the stable door after the horse has bolted.
Next we'll hear is "CrashPlan goes Crash and gets the admins in".
tl;dr: Code42 deleted some of my backups without warning. Their service is unfit for purpose.
"Applications that store data in the application directory should be taken out and shot with extreme prejudice."
You could argue* that for C:\Applications but they silently deleted backups of my network share H:\Applications which has various software that I've purchased, along with documentation and license keys. Not only do I want protection against the software product being dropped, but in some cases this is my only record of my license key. And then there are the programs where it's really important to test compatibility with data files, like .PAF files from 20 years ago.
My biggest concern is that they notified us AFTER THE FACT. If my home insurance company sends me a letter that they will add exclusions to my policy, I'll plan how to manage that risk. If they notify me that said exclusions were added last week, then I'm suddenly exposed to unplanned risk.
Code42 just exposed us all to unplanned risk. They chose both to increase the risk and to give us no opportunity to protect ourselves.
*but please don't argue with me; I didn't write the software. Since this debacle, I found and moved important data out of those directories, but my CrashPlan backups are still lost forever.
This post has been deleted by its author
Received this morning:
"Dear Valued Customer,
On May 6, 2019, our technical services team rolled out a number of changes to the CrashPlan for Small Business data protection service. These changes were intended to make restoring files and machines more efficient by eliminating unnecessary files from your backup sets. Unfortunately, we made two mistakes during this change process.
The first mistake relates to our email notifications sent to you regarding the changes to CrashPlan. Our initial email sent in early April was classified incorrectly as a marketing communication and did not reach customers who opted out of marketing communications. We resent the notification to all customers on May 17, but this did not give enough advance notice to some of our customers. We apologize for this mistake and we can assure you that we have since changed our processes to ensure better communications in the future.
The second mistake involves the actual file changes that we made. As part of this update, we stopped archiving 32 file types and directories. The email notification included a link to an updated list of files that are excluded from CrashPlan backups. One of the file types we began excluding from backups is the .sparseimage file format. We believed that this file format was obsolete because in 2007 Apple introduced a new format called .sparsebundle, which we thought replaced .sparseimage for the use case we track. After we implemented the changes in May, some of our customers made it clear they still have valid use cases for .sparseimage. We now believe we made an error in excluding .sparseimage, and we have since added it back to the list of files we support via backup.
If you use .sparseimage files, there is no action you need to take. The change will automatically be pushed to your client and .sparseimage files will once again be backed up in CrashPlan. This process will take several days as these devices connect to our service. You can see a full list of excluded files here.
We regret the inconveniences that these two mistakes may have caused you. Our priority is to provide a great product that protects your important small business data. We appreciate your feedback and ongoing partnership. Should you have any questions or need assistance, please contact our Support Team.
Thank you for your support and trusting us with your most important information.
Sincerely,
Joe Payne
President and CEO
Code42"
This post has been deleted by its author
This post has been deleted by its author
After almost 5 years of backup (and the money that goes with it, of course), they've deleted 10Tb of data, because of a service that is really not intuitive. They must think that everybody read every little line on their website during their weekends…
There is an option on their app that says « Remove deleted data files : Never », well that does not really work. They should update the text with an « unless this and unless that… and still we might invent new rules that will get your data deleted because we don’t really care ». I got an answer from an employee for that : "The "remove deleted files" setting refers to files that have been deleted from your computer, not to files that have been removed from the file selection." (My fault, I should have guessed that...!!!).
They don't care a bit, you’ve just lost a huge amount of work and their only answer is "sorry next time you should read this and that on our website...
Not to speak of the restoration that is incredibly slow and a software that bugs all the time. Mediocre service!
Well, at least since everything as been deleted, no hesitation to leave now!
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
