Dropbox gets all up in your kernel with Project Infinite. Cue uproar Dropbox is on the defensive after revealing its file-sharing service will in future tap into the very heart of your computer’s operating system. Project Infinite, unveiled in April, will take Dropbox out of the browser on the PC or Macs and integrate it directly with your machine’s local file storage. Items stored in your …
"In Windows, Dropbox will implement Microsoft’s Copy Hook Handlers, COM objects implemented as DLLs and called by the operating system shell. "
So on Windows, Dropbox is actually using a user-space file-system extension. The article is mis-leading on that point.
But yes. Who on earth thought that a file-system at the end of an ADSL link was sufficiently performance critical to require a kernel implementation?
Breath-taking idiocy!
Do you really think the people at Dropbox are idiots? Really? That's slightly idiotic itself.
The performance they're concerned about isn't the upload / download link.
They're building a version of the Dropbox client that virtualises files when your hard disk is full. This is a real problem in business environments where people are running terabyte sized Dropbox accounts on laptops.
The performance they're concerned about is the filesystem overhead of every single file access in the system having to go through the Dropbox code to check first that the file actually exists. Even if the file isn't in the Dropbox folder, the filesystem will have to make a call to the Dropbox extension. You can very quickly make a system laggy that way.
The reason they don't want to use FUSE is that it doesn't really solve the performance problem, as your code still runs in userspace and so requires a context switch, which is very expensive.
Anyway, point being, you may not agree with them for various reasons but they're definitely not idiots. They have non-idiotic reasons for wanting to do this.
I guess the reason is to avoid to sync GBs if not more of data, especially when you local disk has chances of being smaller, and the connection not so fast. If you can just sync metadata, and then read/write file on demand, it looks "faster".
Their idea is probably to work like SMB (or the like) does, you see remote files, they can be cached locally when needed, but you don't get a full sync unless you explicitly require it for some - or it will truly fill your disk if you're an heavy user.
But if the OS and software are well designed, only when trying to access the Dropbox "namespace" code should try to access the remote data. And if metadata are already synced there's really no reason they shouldn't available in user space for a quick check. Sure, if you're going to look into file contents then there will be a lot of kernel calls to access the files - but then the bottleneck will be the network speed and latency, not the context switches - which also are now handled by specific CPU instructions to minimize the overhead. Anyway, context switches are there to protect the kernel from user space code - the overhead is due to access checks and other security features.
Well, yes, I know they are. When it was first implemented they tried to make me pay for it twice and finally finagled me into paying for the extended version. The 'tech' people I talked to thought it was funny I couldn't use it. But, each person has their own opinions, like mouths and other physical appurtenances, some of more use than others. I'm simply not in the mood for vulgarity today. I know I'm an idiot, why display it with monkey-words and monkey-tricks?
Yes, this is designed for others to flame me. Blame my run-in with the VA. No, I am not insulting the author of the comment to which I am replying. However, they definitely are idiots.
CopyHook handlers are just to control what you can do with a item, i.e. deny a copy/move operation - and AFAIK from the Shell only (which means command line operation will go through anyway).
To access remote files on Windows you have two ways:
1) Implement a network redirector. This require a file system driver, and some user space executables (DLLs, services) depending on the needs. That's how, for example, SMB is implemented in Windows. Pro: it allows any file-related API to work. Cons: more difficult to write, and riskier (BSOD or critical vulns), no UI tailoring.
2) Implement a Windows Explorer Shell Namespace extension. That's just some COM userland code. Pro: easier to write, less risky, UI can be tailored. Cons: access only using the Shell API.
For a full support you may need usually to implement both.
More info here:
https://msdn.microsoft.com/windows/hardware/drivers/ifs/what-is-a-network-redirector-
https://msdn.microsoft.com/en-us/library/windows/desktop/cc144095%28v=vs.85%29.aspx
This is what they want to do on OS X too. It seem that Windows has a userspace means of achieving this, but OS X requires a kext. The way to allow/deny filesystem actions on OS X is via kauth. Guess what the 'k' stands for.
I've written a kauth kext. You have a minimal kext that says I'm interested in filesystem actions a/b/c. You get notified when interesting stuff happens and your kext should then inform your userspace component that stuff is happening. The userspace component does the donkey work and lets the kext know whether to allow or deny.
The kext undoubtedly adds a potential source of panics, but the kext should be absolutely minimal: get notification, tell userspace, get response, allow/deny.
Sounds like a lot of risk for too little gain. And..surely they mean they are now using IFS not Copy Hook handlers? Did they try CHH then fall back to IFS? I struggle to see how anything going on on the local machine can be a performance bottleneck if you are getting data over the internet.
Edit: Ah! I get it. With Windows they can use COM, but under Linux they have to hook into the kernel.
Anyway it all sounds too hairy for my tastes. I'd rather they not do it.
But the article is very unclear, it says PC, I assume as a synonym of "A PC running Windows".
... In my vocabulary, a PC is a hardware on which you run the software you wish (unless it is blocked by so-called-"secure"-boot).
And nowhere the article says "Linux". So I will assume again this is only W$ and Mac.
Anyway I don't see such piece of code making it's way into the Linux kernel... and if they manage to convince a distribution to add they piece of code, you just don't use that distribution.
Fuse is much enough for me when I want to backup my files on the cloud (which I do on hubic)
"... In my vocabulary, a PC is a hardware on which you run the software you wish (unless it is blocked by so-called-"secure"-boot)."
Your vocabulary is different to common usage. Since, well, the dawn of time, PC has meant "computer running DOS or Windows" to most people. We know there's a difference technically, but that doesn't change the common usage.
Your vocabulary is different to common usage. Since, well, the dawn of time, PC has meant "computer running DOS or Windows" to most people. We know there's a difference technically, but that doesn't change the common usage.
Doesn't stop me from giving grief to companies that casually mis-use the *correct* definition of "PC", and complain that they are saying "PC" yet have no Linux support. May not get them to port to Linux, but entertaining nevertheless.
Reminds of many (many, many) years ago when there was a series of letters published in Amstrad User I think it was asking what 'CPC' stood for as in 'CPC464' and 'CPC6128'. It took three more publications (three months I guess) before someone wrote in to say that in one corner of the main unit/keyboard were the words 'Colour Personal Computer'.
Sad to say I'd missed that as well despite being a proud owner of a CPC6128 nearly two years.
Copy Hook Handlers are never called by Windows kernel, they are in-process libraries called only in user mode by Windows Explorer (i.e. what purports to be shell under Windows).
The whole uproar seems to be about bastardization of BSD-derived kernel of OSX. In which case, let me be allowed to cite from a comment above "What's wrong with fuse?"
Yeah, that was my thought as well. The Windows version, at least as described in this article, seems like a fairly pedestrian Windows Explorer extension. It sounds like the OSX version is the one with an eyebrow-raising implementation. Aware of its implementation, I wouldn't be very keen to install that either.
Responses like Dropbox's, which we see all the time from vendors, saying how they've run it for however long and "battle tested" it, annoy me greatly. That's absence of evidence that it's exploitable, which is not the same thing as evidence that it's secure. Saying they had it penetration tested and/or externally reviewed would still not be iron-clad proof of security, but would at least better suggest they really understood the potential risks of their design and took serious steps to mitigate them.
Wassat Sherlock? A web browser is not an OS? Heresy!
I have some sympathy for the idea of a kernel-based system, in that we'd expect other remote storage protocols such as iscsi to live in the kernel, and we'd also expect proprietary device drivers to be in the kernel, but (and its a big but!) I think the main issue is that we just don't trust web2.0 companies to behave properly. Would I trust Javascript devs with my kernel?
No.
But then again, cloud storage is not for me either.
Meh, my home PC is only used for games and netflix so they can take all the telemetry they want, my onedrive consists of a bunch of D&D sheets, and my XBox is also just used for gaming.
If any of that gets compromised then oh no, I'll have to reinstall a bunch of unimportant things.
The laptop that I use for work and banking and anything slightly sensitive is set up to distrust everything on the home LAN just as much as it distrusts the internet as there is no need for it to talk to anything else on the LAN.
You don't have to deny yourself nice things in the interest of security, you just need to be smart about how you implement them, same as anything.
Accessing file that you actually want to upload or share is so 2000s. The rest of the files are more interesting and from the browser is rather challenging doing it. It's so inconvenient and expensive with all that patches comming all the time... So let's go for the throat, let's do it in the kernel. The future is for the braves.
Just give me some linux command line tools that allow me to do a remote ls and get/put/sync files please.
Or better yet, ditch your semi-proprietary system and allow people to use scp/sftp to transfer files back and forth with an ssh login to manage them.
Oh wait, thats just like a remote account. Which I've already got. Sorry, what's the USP of Dropbox again?
That's fine for you and the others who can use a linux command line.
Not so good for my father-in-law, though. Most people don't use Linux, don't know what a command line is, and have the barest understanding of computers overall. Any system which fails to take account of this basic fact is flawed by design.
Given that Dropbox is famous for their user experience, I don't see this happening. And rightly so.
"Given that Dropbox is famous for their user experience,"
Uploading files is a user "experience" now is it? Wow, guess I didn't get that memo.
" I don't see this happening. And rightly so."
Why not? I never said it had to be either-or, they could easily do both if they really want to appeal to all. Which seems to be the case if they're developing a linux kernel module.
The whole Dropbox app is more than just "uploading files" clearly. It's a complete user experience that is incredibly straightforward and reasonably intuitive, even for non-technical computer users.
If you disagree, that's fine, but you have to come up with a theory that explains the relative success of Dropbox compared to, say, rsync.
"It's a complete user experience"
Do yourself a favour and knock the buzzword bullshit on the head. You only make yourself sound like an idiot. Using a file syncer is not an "experience", well, not unless you've spent your whole life locked in a box. Have you?
"If you disagree, that's fine, but you have to come up with a theory that explains the relative success of Dropbox compared to, say, rsync."
I'm not saying do command line tools for everyone, just for experienced linux users - which most are. Is this too hard for you to comprehend?
If you're happy using scp/sftp, why even consider Dropbox? The whole point of Dropbox is making internet based file storage really easy to use, so people don't need to learn what rsync is, or have to set up their own file server.
You might as well complain that Cbeebies doesn't contain enough gritty realism for you.
This is where it can get funny. I have Home User or Small Home Business clients who have installed half a dozen different cloud services without even realising.
They get sent one file as a Dropbox link, but then end up signing up for Dropbox in error and installing the application. (Very aggressive website that one, designed to confuse) We then add OneDrive from their Win10 upgrade. Office 365 adds OneDrive for Business.
They then updated iTunes and along came iCloud... so they click on it because their Apple "Genius" told them it would bring world peace. And they then wonder why their copy of Outlook 2010 has now gone mental.
And just to get the full set, they have a Google Drive running - but nothing in it. And no idea where that came from.
Oh yeah... and don't forget the OEM clouds that the laptop makers try and sell. Demos installed and never removed.
Add in a dozen apps that all busily checking online for updates, or "syncing" settings... and it does start to get a little bonkers.
Sometimes it is amazing that computers manage to boot up at all!
Adding to the confusion. If you're on a Windows version that doesn't come pre-installed with OneDrive, the current Office 365 (i.e. Office 2016) installs both the consumer and business versions at the same time!
So you end up with two OneDrive entries:
> "Microsoft OneDrive" (OneDrive.exe) and
> "OneDrive for Business" (GROOVE.EXE).
Both programs have exactly the same app icons as well!
I'm running the Dropbox client for Linux on my machine right now, no bowser used. It runs as a user and synchronises the contents of a folder in my filesystem.
There is a 'Dropbox for Caja' (super-duper full integration, it claims) that asks for Administrator rights but I don't use that one. The one I do use has Dropbox integration options in the context menu. However, it always sets itself to run at startup, so I only run it when I need it and kill its startup setting if I can be bothered. They are sneaky and seem to be getting even more sneaky, with no need or benefit to the user.
"I'm running the Dropbox client for Linux on my machine right now, no bowser used. It runs as a user and synchronises the contents of a folder in my filesystem."
Perhaps I'm just old fashioned and not hip enough, but I don't want a background process automatically sync'ing files to a remote site. And if you have to ask why then you probably won't understand the answer. Any linux user worth their logon can write a script to automatically sync given the correct command line tools if they need that functionality.
This just seems like the next logical step, which I would expect from them.
I haven't used them in years. Can't use it at work -- IT dept won't allow it -- and the integration on my home machine was irritating.
Sure, there are risks involved, but for those who use Dropbox to begin with, blurring the lines between systems is nothing new. Worth the risk to many. And they'll just suss out the fixes as problems arise.
Just like all the other SW that's wedged in there nice 'n' tight.
And again it is the poor sods that are going to suffer. Sure, a bunch of nerds, geeks and commentards cry out in dismay and declare a boycot, but the average user will just click 'ok' when the pop-up dialog promises them faster dropbox when they upgrade and have their machine owned by yet another yankee data guzzler. I pity those who don't have a clue!
“We’ve been running this kernel extension internally at Dropbox for almost a year and have battle-tested its stability and integrity,” Dropbox wrote, adding: “After careful design and consideration, we concluded that this kernel extension is the smallest and therefore most secure surface through which we can deliver Project Infinite.”
The statement continued: “We understand the concerns around this type of implementation, and our solution takes into consideration the security and stability of our users’ experience, while providing what we believe will be a really useful feature.” ®
Pardon me if your "consideration of security" does not convince me. Of course you would believe it to be useful feature. I don't.
That extension won't get anywhere near my kernel. Why, indeed I don't trust you and you won't get kernel access to my machine.
If you don't trust them, why are you giving them access to your files in the first place?
I never said if I did or not.
As it happens, I did use them long time ago to copy/share some encrypted files across different platforms.
That is quite different from the trust required to give them kernel access.
"We’ve been running this kernel extension internally at Dropbox for almost a year and have battle-tested its stability and integrity"
"Battle tested" on corporate computers that have controlled configurations... I doubt they have hundreds of thousands of differing hardware and software configurations let alone differing patch levels.
Given that most hardware makers already have unfettered kernel access for their drivers, I don't see that this is too much of a problem. DropBox is likely to write much tighter code than most of them. RealTek audio, anyone?
Besides, Dropbox already have access to everything of worth on my computer. That's kind of the point.
What more have I got to lose than if hackers simply hacked the existing Dropbox client, which would already be the worst possible outcome?
In which case, sir, you are an idiot. There is a massive difference between hardware kexts and a data copy agent. Even if they "hacked the client", that's a user process, not a kernel one.
Currently, Dropbox has access to one folder on my system. The only non-trival item in that folder is an encrypted .dmg, to which Dropbox does not and will not have the key.
Anyone who gives an online syncing service "access to everything of worth on my computer" deserves everything they get.
I'm not an idiot. I just don't feel the need to wear a tinfoil hat. Unlike you, clearly, with your encrypted dmg file.
Which defeats some of the best features of Dropbox, I might add. Every time you change anything in your dmg file, the whole thing gets synchronised. That means you lose file-by-file versioning. You also lose file-by-file synchronisation. If you have one file open on one computer and another open on a second computer, you have the entire .dmg file locked on both machines. That will mean that, at the best, you'll end up with a conflicted copy duplicate of the entire thing. At worst, you run the risk of overwriting changes.
And I'm the idiot?
There is a massive difference between hardware kexts and a data copy agent
Not really - once you've got kernel context it doesn't really matter what you say you are going to do with it. At least you know Dropbox is going to have access to your data and can choose not to use it. Do you really know what your drivers are doing?
Are not kernel extensions. They're vanilla user-space part of the Windows API. I'm frankly pretty surprised that Dropbox isn't using them already actually. Nothing remotely unusual about using cooy hook handlers.
Also, this article talks about moving out of the web. Dropbox is already out of the web. It provides Native applications which run on the desktop in the background, and it always has.
I use Dropbox as a first port of call backup thingy as it's useful when one is on the road.
I use SyncBackFree to determine which folders I want to have backed up and that copies files to the Dropbox folders on schedule and that's it. No dicking around with browsers (unless I need something) and then that's sufficient for my needs.
There is no way on god's earth that a lot of large companies will allow this to be used on a system that also contains their data. i.e. Not just the company owned systems, but also the personal systems that just have incidental use business data on them.
Unfortunately this is another great idea that has built a head of steam in a small group in the valley without understanding how the real world will respond.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
