I used a Tegra K1 Chromebook with Ubuntu and it kicked ass
I used a Tegra K1 chromebook (with Chrubuntu on it) and it kicked ass. (Acer Chromebook 13, I got the variant with 4GB since I knew darn well 2GB wouldn't be enough and they don't make an 8GB or more variant.)
The ARM was reasonably fast, I got 22 hour battery life under "normal" use and 12 hours with all 4 cores pegged out doing video encodes and whatever. The Chrubuntu install copied over power management scripts from the ChromeOS side so it automatically would drop to the 5th "low power" when usage got under roughly 700mhz on a single core. (The odd side effect, the CPU usage didn't scale, so it'd almost always claim 90-100% CPU usage, it'd just be 90% clocked at like 50mhz if it wasn't doing much. If you installed a CPU meter it'd do that in chromeos too; I just looked at clock speed instead if I wanted to determine how loaded up it really was. Only negative effect of the setup was the canned low battery "warnings" at 20%, 15%, 10%, and 5%... getting warned when you have like 5 hours of battery life left is of course a bit daft. My current KDE install (on a regular notebook) does let you change that, but the battery manager thing in the desktop I had on there didn't as far as I know. Pretty minor nuisance though, I'd just dismiss the warning and ignore the red battery symbol.)
The GPU was roughly dead even with a GTX650, but the whole chip used like 10W of power max (GPU + CPU) rather than the like 65W the GTX650 used. At the time I had a Sandy Bridge desktop, and the laptop would video encode at like 2-3x the speed the desktop would. Having a modern, fast ARM system would be fabulous! I didn't run a bunch of other benchmarks, just found that even heavier tasks (as long as they weren't memory-intensive) ran great on it. Drivers? Nvidia driver, including Vulkan and CUDA, at least for Linux they've had Nvidia ARM drivers for many years. (So I imagine the driver has no Intel-specific code baked into it and can be trivially brought up for ARM on Windows as well.)