Why one storage admin fears Justin Bieber What are storage admins afraid of? We imagine failed backups, wee-hours SMS alerts and vendor maintenance bills are high on the list, with rampant data growth a constant low-level worry. Warbling teen pop idols? Probably not so much. Unless the storage admin is Dean Downes, the Chief Information Officer for St Aidans Anglican …
Maybe it wasn't an ad from El Reg's PoV (you probably didn't get paid) but I imagine the sysadmin would have done for shameless publicity like this!
Though I must say, I do feel sorry for the guy - at least there's nothing like this in my line of work... though I am expecting our Internet bandwidth to go to shit over the next two weeks!
I wrote a program that runs each evening. It goes through the user areas, hashing each file (less than 10MB) and comparing to a blacklist. The blacklist contains the hashes of every illegal music file, game, piece of pornography, topless picture of the Twilight hunks, lolcat, hacking tool and general fluff we have deleted. We occasionally have a through rummage of student areas for things, and the program ensures that it stays gone: If a student brings the offending file back via USB stick or downloads it again, it'll vanish overnight.
Let's say 1,000 students each with 1,000,000 files ~= 2^30 files. The rough probability of a collision happening at random with 2^30 files and a 128-bit hash (e.g. MD5) is 2^-67, or 128,000,000,000,000,000,000:1.
That's roughly the same probability as winning the National Lottery three times in a row.
I assume there was no procedure :)
Class-centric virtual servers and quotas.
Those being taught how to shoot and edit video get space on the videography-class server, everyone else gets enough for RTF versions of their history essays and nothing more.
How many miliions of characters are the kids typing?
My kids' school actually gave out usb sticks. That seems like a decent solution, at $6/child it saves all the complexity of creating and maintain some SAN and makes the kids responsible for their work. Plug them into terminals running ISO-based XFCE with LibreOffice and off you go. Mail size limits can be enforced at the server to stop email being used as a storage cupboard.
Oh how at school I loved to screw with people like you.
Well done, you trained a school of kids now trained in the art of moving pornography without being noticed.
Whats that, you hash the file when its a jpeg? Shame I inserted the image into a .doc. Why would anyone check a .doc called "version12345.doc".
How about instead of messing around with silly tools like this; you give the students X space, and limit them to that. Anyone needing more than X space has to apply to the sysadmin to explain why they need the extra space. In return they can be educated on what costs that space incurs and so they learn to treat it respectfully.
Nah, that'd be like school or something....
"It's utterly trivial to defeat hash-based searches "
Not for kids it isn't. Firstly, they're lazy; secondly, they're pretty dumb - that's why they're in school; thirdly, they have access to all the porn they can eat at home these days so it's not worth the time to defeat systems like this.
Kids today are growing up "knowing" about compuers and IT in the way that previous generations have grown up "knowing" about cars and mechanics, ie they're mostly uninterested users with no real technical skill at all. I doubt more than 5% know what a hash is.
My dog ate my homework becomes "the sysadmin deleted my homework" (because I was smart enough to stuff the .doc files with loads of high res pics that I scaled down).
Oh, you think they won't? It takes exactly *one* to have an above average smart brother or a dad with a sense of humour who knows his way around IT and the news will spread faster than you can swear - if there is one thing schoolgirls do well it is distribute information..
Funny, all the schools & colleges I worked at used a fixed size for user directories, & it took a very persuasive arguement on the part of the user, plus an examination of the current contents of thier user area befor the size ws increased. This was usually only done for year 11 & the sixth form. Worked a treat.
Yes; those are the kids that the smart kids refer to as 'decoys'. The smart kids will only tell a select few of workarounds, in the sure knowledge that you'll be too busy investigating dumbasses to play close attention to what the smart kids are getting away with (ie murder).
Also: They probably know your passwords. Against all probability and reason.
Back when I was at school, the guy who ran IT regularly came to US to fix problems, because we knew every inch and KB of the network far better than he did. He just worked there; it was our plaything.
Back when I was at school, just before the widespread adoption of microprocessors (just after the invention of the wheel) we went to the local college to use the mini-computer (with 8 x ASR33 terminals, happy days). If we wanted to nip down to the NUS bar for some underage drinking we would call a program we'd hidden in the executive to crash the system. The admins never sussed it out.
I thought this was going to be an interesting article about how a school's sysadmin had built a new, easily expandable, storage system using off the shelf commodity parts. Or maybe an article about how the school has employed some new dedupe software that's really good at reducing the size of Justin Bieber videos. Instead it's just an overview of a Dell case study.
This post has been deleted by its author
I would say that this sort of data scenario, where the users have no incentive to do any housekeeping, is exactly why you need a strict policy of grooming. Except that's probably not a word you want to be using in a girl's school.
And what ever happened to the concept of disk quotas? Doesn't anyone use them anymore?
It goes like this:
1. No, we won't sell you what you want / need
2. We are happy to sell you something completely innapropriate
3. No, we won't take your money
4. no, really, we don't want it, we have plenty, thanks
5. oh all right.
6. OH YOU WANT SERVICE? that's extra!
7. What, you mean actually REAL service, not 'service'?
8. at your premises? do you have any idea how big this country is?
9. Bloody government, hindering business in Austfailia, people buy stuff from overseas because they don't trust local vendors, and the local vendors add 300% because they have swimming pools and chryslers and fur coats for their trophy wives to buy... (now proven by CHOICE).
Where I worked there is no concept of quotas being used to enforce capacity management. Using Quest Storage Suite I managed to encourage users to free up 3TB of space that shouldn't have been consumed in the manner they consumed it.
Problem is that Service Management wont enforce a policy on the business for acceptable use.
So let's call that 300GB every term. There are 3 terms (presumably), so that's about 1TB/year. Let's make it redundant, so call it 2TB/year. That's about £100/year in storage growth. And he needs a _SAN_ to manage that? Seriously? Is that before or after deduplication? Considering most of the content that ends up getting shared is going to be the same for every user involved, deduplication should reduce the data growth by a large factor.
But seriously, we are talking about adding 2 1TB disks during the summer holidays every year. With something like a small server and a HP MSA70 disk tray (25 SAS/SATA disk slots, £300 or so for a refurb, i.e. sweet FA), that's 12 years worth of storage, before he might start to upgrade the oldest disks (not including upgrades due to disk failures). Something like that with ZFS on it would be dirt cheap, and most certainly wouldn't involve shelling out for a SAN or waiting 11 weeks for the vendor to "install it". But that would require some DIY competence. How do these people get to be in the jobs they are in?
huh?
quota's, dedupe, copyight scans all available and best practice, no lets buy more storage from a company that doesn't want to sell it to you or install it. Way to go. My experience with EMC was much better, due to corporate restrictions they had to install, 2 days after it arrived, its worked fine since.
If its on their network, its their responsibility to pay RI Ass the fines, if you don't actually positively discourage it I imagine their lawyers will eat the college up, this guy has just written his resignation letter.
good luck.
Sounds like typical Dell service to me. The fuckers sold us a completely inappropriate SAN and FC switches, then "forgot" to mention that we had to pay another couple of thousand dollars to license the ports on the switches. Not to mention they then completely fucked up the installation to the extent that we eventually ripped the whole thing out and put another manufacturer's kit in instead.
Disk deduping - 500 copies of his mugshot use slightly more space than 1 copy.
ZFS is nice for this.
As for Dell - no surprises about ths saga - the only screaming question is WHY the school continues to do business with them when any number of other vendors will be falling over themselves to both sell things and match or better the prices.
"Downes wishes they'd think again.“SD looks just as good on a PC screen,” he said at the event." - Which PC screen? A grey box laptop from 1997?
Obviouslyif you play it in its native size it will "look just as good" but what happens if you want it full screen?
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
