When the IT department speaks, users listen. Or face the consequences

Friday is here! A chance to slope off early, enjoy a few brews and look back on a week of hard work. Unless, of course, you are one of the unfortunate souls destined to be forever at the beck and call of users. Bask in the fact that it's not you as you peruse this latest instalment of On Call. Today's tale comes from a …

[swampdog]

My Acer Aspire 5720 laptop came with Vista on it. It lasted 3 days before it got centos'd. It was months before I realised the fan no longer worked but a hacky firmware update partially fixed that. Darn thing still works. Linux mint now. Sits at the end of my bed as basically a "tv/audiobook". It's seen Vista/Centos/XP/2003 server/Win7/Mint. Never gets turned off but bugger me if last time I dragged it outside to plug into my car odb2 port the bugger lasted almost 20 minutes (original battery).

Contrast that with my desktop PC. Mobo is packing up - hd access is on a par with ethernet. They don't make sh*t like they used to. :-(

[Kiwi]

Contrast that with my desktop PC. Mobo is packing up - hd access is on a par with ethernet. They don't make sh*t like they used to. :-(

Shit SATA cables. Doesn't matter if you brought a box of 100 for a buck of paid $100 for one. I've seen a lot of HDD issues resolved by a new (even an el cheapo!) SATA cable.

Meanwhile, I have PATA cables that are decades old, been in and out of dozens of machines/drives, and still 100% reliable (at least when the mobo has a suitable header!)

[Boris the Cockroach]

I'm pretty sure /dev/null/ is between some users ears

<<goes away muttering about telling manglement about an impeding hard drive failure and then trying to recover data from said failed hard drive 1 week later while manglement scream "WE NEED THAT DATA"

[404]

I managed to skip Windows ME - went from 98SE to Windows 2000 - thank jebus!

I do remember all the screaming though lmao...

[Tigra 07]

"one year, every month"

Clearly OP is a time traveller... Which pocket has my sonic screwdriver?

[Venerable and Fragrant Wind of Change]

Yeah, I took that for a figure of speech :)

[John Brown (no body)]

In IT support, often a month IS a year! (Or at least feels like it)

[Anonymous Coward]

A modern update

We have a policy that clerly states that files should not be kept on the C: drive for any length of time. i.e. if you're out of the office and can't connect back it's ok but you must sync the files to the network when your return.

User had a problem with his laptop, it wouldn't connect to the network drives. He chose not to report this and started saving all files on C. Then he clicks on a phish and the laptop gets hit by ransomware. Luckily for the business his network problems prevented any spread, unluckily for him he lost everything on the laptop.

Unlike the OP I didn't laugh in his face and walk off.... he was subject to disciplinary action for breach of policy and loss of important data.

[Pascal Monett]

Ah, finally a happy ending.

[tfewster]

Re: A modern update

That's a true BOFH - ensure the problem doesn't re-occur by removing the point of failure (the user).

However, I have to say:

- Did the Helldesk not re-remind the user to ensure all their files were copied to the network before the nuke & pave? Yes, they'd been told repeatedly, and the disk errors may have made it too late already, but it saves the inevitable fallout with Manglement. Unless "Philippe" really wanted to make a point...

- Did "Philippe" not get an eye-wateringly expensive quote for data recovery to further illustrate the point? Chances are that a fresh install wouldn't have overwritten the space that had user data on it.

[PickledAardvark]

Re: A modern update

I was called in to setup a new Mac PowerBook after the owner, a hospital doctor and researcher, had allowed the previous one to be stolen. His normally gruff assistant beamed with delight while she explained the disciplinary action he faced for losing sensitive patient data.

In my experience, there are two sorts of doctor: Bright people who assume that they are good at everything, and really bright people who understand that they aren't an expert in every field.

[Tim99]

Re: A modern update

Yes, medicos can be like that - Lawyers seem to be even worse.

[swampdog]

Re: A modern update

Lawyers don't understand money. They just assume they can invoice for more. Earlier in the year I had to explain to a GP what the "VW diesel scandal was". This does exceed the abilities of at least one plod though, who after I had a motorcycle accident, could not grasp the concept of a sequential gearbox.

[Anonymous Coward]

Piss poor IT management

How hard is it to insert a step before the "reformat drive" where the support dude he sent makes a quick copy of My Documents? You know not everyone is going to obey the edict, and the files will be lost if the disk completely fails, but a lot of the time they'll be recoverable and good support involves actually providing the best service to your users, not being a jerk and saying "told you so".

[Anonymous Coward]

Re: Piss poor IT management

providing the best service to your users!!!

ahahah not in this ivory tower. you will obey my rule!

[Anonymous Coward]

Re: Piss poor IT management

If the computer won't boot, or the hard drive itself has problems (bad FAT, physical damage, etc.), how would you intend to copy My Documents? If it simply won't boot, you could pull the drive... oh, wait, full-drive encryption...

[Martin an gof]

Re: Piss poor IT management

If the computer won't boot, or the hard drive itself has problems (bad FAT, physical damage, etc.), how would you intend to copy My Documents?

By that count, how would you intend to reformat / reimage? Under those circumstances there's no way - short of specialist services if you are very lucky - to get the data back. The drive is disposed of and a new one fitted. However, in TOA,

Philippe "sent a support dude who promptly reformatted her drive during the evening and reinstalled all for the next day.

which implies that the drive was actually ok and with a little care, files could have been recovered.

It's one thing to teach a user a gentle lesson, but real life means that outside of Simon's writings we shouldn't be vindictive.

Like the time a user presented me with a non-working 3½" floppy disc containing the only copy of some (to her) important files. Turned out that not only had she spilled hot chocolate on the thing, but had then left it in the desk drawer for a week, drying out.

One sacrificial case and a gentle wash under the kitchen tap later and IIRC I recovered all except one of those files. Gentle lesson taught (one file to recreate manually), genuine service rendered (my reputation enhanced) and, of course, six months later it's all forgotten :-/

M.

[teamonster]

Re: Piss poor IT management

This is assuming that the user will know to inform the IT guy that they haven't followed company policy. The IT bod will just turn up, maybe when the user is not there, and assume that they have followed protocol and all their data is safely up on the server.

Also, if the user hasn't followed protocol, does the IT bod have the time or the resources to start data recovery on the drive? This at minimum would involve some kind of boot disk - assuming the drive was still working - and then trawling the drive for personal files in wherever location the user had decided to stash them. What if you don't find them all? Is the IT Dept then held liable for not recovering them?

Once you teach users that not following instructions has no consequences, then you might as well not bother with any kind of IT policy. As has been illustrated elsewhere in this article's comments, even extreme catastrophe won't teach some of these morons the error of their ways. Lessons will not be learned and thanks will not be given. You are just making more work for yourself.

[Martin an gof]

Re: Piss poor IT management

Lessons will not be learned and thanks will not be given.

I take your point, and I've never worked in a support role in an organisation large enough that I don't know each user quite well, but there is a dividing line - somewhere - between efficiency and following the rules, and vindictiveness, and in my limited experience many of these situations start out with a user popping up and uttering the immortal line "it's the only copy of that file and I need it for a meeting tomorrow", so you will already know or suspect that there isn't a network copy.

Obviously there are complications if encryption is in place, but without that, simply slotting the old drive into a USB interface and firing up quick-and-easy tools such as PhotoRec, GParted or even SpinRite can recover quite a lot. dd first if you want to be extra safe.

It is also my limited experience that the sort of user who still doesn't take notice of "best practice" after nearly losing files, won't change their ways even if they actually lose files. This is why (as others have pointed out) redirected home folders are a boon. They do have their own issues, but making as sure as possible that all data is kept on a network file server takes file security responsibility away from the user and plonks it firmly in the lap of IT, who should know what they are doing.

In the meantime, even a small number of files recovered from a dying HDD (or USB stick or SD card) can turn someone's day from utter disaster to survivable. SD cards are a particular example as unless you own a high-end camera which takes two cards and mirrors them, or a clever camera / phone which automatically syncs to a cloud service (roaming data charges aside), most people will spend an entire holiday relying on their memories being kept safely in a small easily-damaged or lost sliver of plastic, metal and Silicon.

M.

[Anonymous Coward]

Re: Piss poor IT management

Like I said you can't recover them in all cases, but you should at least try. Not saying you need to go to great lengths, but when the support guy booted off a CD/network for the reinstall he should have at least tried to access My Documents, and if he was able to do so should have taken a few minutes to copy them off safely to the network or another drive.

This "well I warned them many times so it is 100% their fault" is like if someone is changing lanes where your car is because they didn't look, and you refuse to hit your brakes or move out of the way and instead let them hit you. It is 100% their fault, and they will have to fix your car, so why not?

[[e-mail]]

Re: Piss poor IT management

DougS, if the hardware is dead then there is no way the "support dude (he) sent" can make a quick copy of anything.

Stop making allowances for people who break the rules. Rules are there for a reason. This is why the world is in the state it is in - some rules exist to make the world as pleasant as possible for the greatest number and can be flouted with impunity but some - like the fact that you [I]cannot[/I] recover files from a broken HDD - are simply impossible to break, bend, coerce, persuade or get around just because you don't think it should apply to *you* (where 'you' is the User, not DougS necessarily).

Sometimes the "best service to your users" is to enforce the rules and not keep allowing some people to ignore them - most people can do it, what makes the others so fricking special? It's a bit like "punishing" kids who steal cars by giving them free trips to theme parks and sending them to adult education classes in car maintenance (while the kids who *do* manage not to break the law get absolutely nothing and the adults on those courses have to pay for the lessons) - my school started doing this the year after I left because some bleeding heart idiot thought it would make the young criminals less likely to repeat their offences... Guess what? It didn't, and those "people" are still causing grief for everybody else.

[Doctor Syntax]

Re: Piss poor IT management

How hard is it to insert a step before the "reformat drive" where the support dude he sent makes a quick copy of My Documents?

It depends on circumstances. If the user is screaming to get the PC back in action they're not going to be happy if you start booting up a recovery CD to get the files off.

[Jou (Mxyzptlk)]

Re: Piss poor IT management

[ ] You read the article.

[ ] You understand the concept of "disk errors".

[X] You are exactly the type who causes such problems, and on top blaming others on your failure(s).

[teamonster]

Re: Piss poor IT management

I have been in a situation where I have been a visiting tech at a remote office. I have walked in, set up and then and had to field local user complaints, as well as dealing with a variety of server issues that would take up a lot of time. I was the only tech guy at an office of about 100 people. If a user presented with "I can't follow basic instructions" I didn't have the time to do a search on their drive for all .doc and .xls files wherever they might have stashed them.

[Jenny with the Axe]

"Your backup routines suck!"

Back in 1997 I was working helpdesk for an organisation where a lot of users still had Windows 3.11. One of them called in and asked us to do a restore of a document that she had stored on a network drive.

I asked for the name of the document and the user wasn't sure. I asked when it was created, and the user told me:

* User had created the document that day and worked on it for several hours, carefully saving it often.

* Once User was finished, they closed it, They then decided to rename it

* User clicked on the document in Windows Explorer and marked the entire filename, including the ".doc" extension. They wrote the new name and pressed Enter.

* Then the file lost the "picture of the paper with the three little lines" . In other words, since User removed the ".doc" extension, Windows no longer associated it with a known file type and so the icon meaning "I know what this is" changed to a blank one (I think - like I said, this is more than 20 years ago).

* User got scared and deleted the file. And then called asking me to restore it.

The only problem was, even though we *did* do backups of files on network drives, those backups were made during the night... And documents on network drives did not end up in any Trash folder or had any other sort of recovery mechanism.

The user was not happy.

[Anonymous Coward]

Re: "Your backup routines suck!"

I think you mean the user was not educated, and happiness doesn't matter. Also, that education would have been from you, in IT.

[swampdog]

Re: "Your backup routines suck!"

I think it means a lot of people do this else M$ would not hide file extenders by default.

Not that I agree with it. Bad training. The reader can bring to mind their own car analogy.

[Doctor Syntax]

Re: "Your backup routines suck!"

"I think you mean the user was not educated"

Previously. After the event they were.

[ibmalone]

Had the opposite problem when I started my PhD. I liked to listen to music while working and so, rather than keep lots of CDs in the lab and have to keep changing, gradually ripped them to ogg onto my desktop machine. Thought I'd be organised and put them in My Music on the computer. What I'd never encountered before, this being the first time I'd really used a machine on a domain and it being XP era, was roaming profiles. Sometime in the first couple of months someone had a word with my supervisor and then my supervisor had a word with me about GB of music on the network and backed-up...

Somewhere I've still got two DVDs full of ogg vorbis that I burned later that week. Though in hindsight splitting them alphabetically by artist was a mistake.

[Jou (Mxyzptlk)]

Thumbs up for "first time ... roaming profiles"

[Stu_The_Jock]

Data ? Backup ? No need

Previously had the "pleasure" to support PCs for a chain of pharmacies. All machines had a build in back-up system, which copied all patient prescriptions. Later model PCs had 2 hard drives, and the store manager was told clearly to run the daily backup from the desktop icon every day at close of business, and the weekly one friday evening or saturday. (Daily took 30 seconds, weekly about 10 minutes as ig made a full hard drive snapshot.)

The usual call for a non-booting PC comes in on a Tuesday, drive had failed, no problem, we'll restore from Friday's image and then add in yesterday's data set.... Pharmacist looks slightly pale. Fire up recovery system, last weekly backup was from when we installed it 30 months earlier, not great, he'll just have to let al, the software updates since sort themselves out.

Last daily backup, 24 months earlier.... he then has the choi e to write off 2 years of prescription data, or buy back the faulty drive to send off for data recovery at his own cost..

[Angry IT Monkey]

I worked at a place where several managers left their PCs on every Thursday night because they believed local drives were imaged to "the server" weekly.

I never found this mystical server with enough space to hold a 1:1 copy of every local drive but I did put in an estimate to build one.

My suspicion is someone high up had this hare-brained idea and the IT Mangler played along knowing it was less painful than explaining why it's ridiculous (sending that much traffic over a 100Mb network in a 24x7 operation for a start)

You learn to image disks and for some "special" users you screenshot the desktop because if the icons are rearranged they complain it's broken and refuse to use it.

Happy days!

[KittenHuffer]

That's when you move all their icons off their desktop, move the task bar to the top of the screen and set it to auto-hide, and then set their desktop image to the saved image of their original desktop!!!!!

[GrumpenKraut]

> You learn to image disks and for some "special" users you screenshot the desktop because if the icons are rearranged they complain it's broken and refuse to use it.

Reminds me of the guy who got into a screaming rage because the new browser (version) didn't have the default font set to what he was used to: "IT IS BROKEN!" "I CANNOT WORK!" "JUST FIX IT!".

I was tempted to superglue his face to the monitor.

[gnasher729]

How big are your local drives? If you have say 40 Macs for normal office use, £2,000 buys you a Mac Mini, three 14TB drives, giving you 1TB per Mac. Turn on Time Machine and everything is backed up all the time. Files a user lost a year ago and that you miss just now can be found.

[defiler]

Storing in the wrong place?

Literally today I've had that old chestnut of <user's mailbox is getting full>, user requests more space, I take a quick look at his mailbox and 2/3 of it is in "Deleted Items"... I suggest emptying that, and am faced with the response that he's "keen to keep hold of his deleted items for now"...

Fine - he has a cost centre, and I'll tell them the cost.

Do I win a beer? Please?

[404]

Re: Storing in the wrong place?

Have a beer->

Had a guy who used his Outlook as a file system - over a hundred folders and over 10GB of a .pst file - on Windows XP... and he wondered why his laptop was so slow to start up with Outlook being in his start folder...

[Kimo]

Re: Storing in the wrong place?

I was once given the task of telling a list of users to clean out their inboxes before a major maintenance on an Outlook server. They estimated rebuilding the system after would take about 1 hour/gigabyte, with a handful of accounts nearing the 3GB storage limit. As I was not a senior position, most just told me "no" even knowing that we were looking at an extended email outage.

[steviebuk]

Not me!

"Let's face it, IT teams got pretty adept at the "nuke from orbit" approach with Windows – a skill that has stood them in good stead to this very day."

I hate having to do that unless really is the only option. I'd prefer to find a fix. The best fix I found, which I keep spouting out, was back when laptops were freezing when going to network drives after login. Would last for about 5mins. Not every laptop user was being hit. Stupid solution from management was "Rebuild the laptop". But that took time, even a pre-built one, they'd still have to setup everything again like mail etc, they claimed "Its quicker than finding a solution". I argued to be given time to investigate but was told "No" which pissed me off no end. I'd quietly investigate myself for a few mins when I'd find one but really needed time to sit down and look properly. Eventually my manager, giving up with my constant asking was going on leave and finally had been hit with the same issue. She agreed for me to look at hers. Took me 30mins and I found a fix that took all of 1min, compared to the rebuild that took at least 1hr with encryption and about 30mins even if pre-built.

You'd see explorer running at 50% in Task Manager which is shit. Use Process Explorer and you could see the same, but then you could see what .dlls were being loaded in the Treads section. And there it was, a .dll running at 50%. Kill that and the freeze stopped. What was it? Related to the encryption. Was it important? No, it appeared to be just a .dll that searched the network for encrypted documents so it could change the icon for them. We didn't encrypt individual documents so was pointless. Any machine that then had the issue I'd unregister the .dll, problem solved (thinking about it now, I could of requested 3rd line push it out in a group policy but never bothered).

The one similar to the On Call story was at the NHS. A notoriously difficult director who, for some reason, I managed to get on OK with. One day I saw that she was saving ALL her files to her desktop. I warned her none of those were being backed up, you need to move them. I even created a folder for her so she could drag all the files to it later. Then I left for another outstation for a week. Came back for one day with it all kicking off. She had raised an angry call with IT over a deleted file she couldn't recover and was being a dick to the IT team that was based there. Then she saw me in the kitchen and I just smiled and said "I did warn you". She calmed down after that as she knew it was her own fault and IT weren't to blame.

[Daedalus]

Comme d'habitude

Seriously I saw no BOFH-ness here, just French-ness.

But the true French response to "I've lost all my files" would have been le vieux shrug and a low-key "bfff!".

[Callown User]

Re: Comme d'habitude

My reaction on reading the article reminded me of my days (years) as an European product manager for products and processes, each nationality had it's own funny ways, however the French developed a very unique way of working regardless of the pain they would suffer.

PS in spite of this I still voted Remain !

[Anonymous Coward]

I've had the reverse experience. In a previous job, I used to diligently save files to the shared drive on the server, as instructed by the IT dept.

Then one day, the server RAID controller failed, and it turned out that the "regular backups" that the IT team had scheduled had not been working for the past several days either, so I lost several days work.

The IT guy just shrugged and said there was nothing he could do. Meanwhile, the people who had ignored the instruction to save files to the server carried on working on their local hard drives as if nothing had happened.....

These days, I make my own backups of vital data.

[ma1010]

Sometimes they can learn

Back in the early 90's, before our company had a network at all, I had a boss who loved to save everything on floppies because she moved between work and home computers. I told her she should save on the HD and copy to floppy for backup/transport, but she didn't listen and kept saving to floppies. Then one day she had a 400 page critical file she was working on refuse to open in WordPerfect.

I fired up some of the good old-time Norton Utilities and managed to get it to open in WP, losing only about 3 pages or so. I repeated my advice about using the HD as her primary save location and using floppies for backup/transport. She listened that time.

[Lopan]

I had a similar thing happen. The boss of one of the other departments was told to backup your files on a "z" drive and a external drive and was told multiple times how to. Didn't and lost everything. Told my boss about it and he asked how much to send out the drive to be recovered. Told him and he turned white. No recovery.She hated my guts but couldn't complain. Idiots happen everywhere. I retired after she left.

[G7mzh]

We had a few customers like that. When they found out how much it would cost to recover, somehow their data always became less important!

[The Oncoming Scorn]

Flash Back Or Dormouse Corner

A earlier story retold.....

..the top dog legal secretary, had her desktop completely (Literally not a inch of desktop real estate, with folders as close as they could be without being dragged\dropped into its neighbour) filled with folder of every single contract that was ongoing or kept for legacy\retrieval.

A single HDD failure would have effectively wiped out this former subsidary company of Brutish Gasoline, ohhh boy she really didn't like hearing that & refused the new hardware possibly over the time it would take to copy everything to the network & then create nice shortcuts on her desktop because "I" might lose everything.

Even earlier in Zummerset County Council:

1: Delivering a reimaged laptop back to user.

"I'm terribly sorry but your old HD was toast & we were unable to recover the data."

"All the stuff I was working on for 9 months is gone:"

"I'm afraid so."

"You got it all back last time."

"On that occasion you were lucky & you would have been reminded about the importance of backups just in case you hadn't realised that for yourself."

"You stay here....You can explain why you have lost my data to my boss."

I had other calls elsewhere across the region & used discretion as the better part of cowardice. With hindsight perhaps I should have stayed, enjoyed as I explained all to the Ladies boss.

2: The time I forgot\missed the migration of a PST to a new machine...customer signed off as all good. On the sixth day she noticed (Outside the grace period) but could we find the machine in the piles upon piles of machines we were wiping for disposal - Nope.

My chain of events, l dropped off the machine from site, went to lunch, came back business as normal.

The chain of events, dropped off the machine from site, went to lunch, sleepy Geoff AKA "Dormouse" decided he would quite like a desktop machine on his desk for his browsing & ticket updates, picked up machine, reimaged & used it in his little corner.

We found alas it too late for the PST, but were able to cite the 5 day grace period in our defence.

[G7mzh]

Where I used to work, every employee had their own storage area on the network, mapped as "Drive N:" on their PCs. It also had a shared directory used for things like software updates, so people couid run the update when they weren't otherwise busy.

Despite it being frequently mentioned, I think only two or three of us actually used it for personal data.

[jelabarre59]

Cloud vs floppy

At least at our job we have a SpiderOak setup that does a regularly-scheduled backup of files on the laptop. We're not talking beginning/end of the day here, more like some number of minutes. Of course, I'm paranoid enough that some projects are kept in Git or even Google Drive.

This is in contrast to a law office I used to do side-work for. They had heard about the risks of hard-drive crashes (this was the early-mid 1990's mind you) so they would keep their files on floppies instead... Yeah. I kept pushing them to set up a proper file server with regular backup instead. Eventually I moved out of the area, so I don't know if they ever followed through.

[Tom 35]

Work from home sales - BYOD

Stored all his docs in folders on his desktop.

Deleted all the files because the drive was full

Ran defrag because it was still slow.

Discovered he NEEDED those files. Came into the office so I could un-delete them LOL. Told him no, maybe the CIA could recover them, but not me. Told the sales manager the same thing. I think he asked everyone in the office, it was his idea to do no control BYOD to save money after all.

[OzBob]

Er, happened to me last month

when my fellow techs reformatted my "thick client" and assured me "my documents" was backed up to the network share. Sure it was, but they must have selected the "resync from scratch" option on the build because it wiped everything I had. And we did not have backups. Now I have a separate Drive Mapping I use for storing documents (plus a wiki I have created for all my documentation). This would never happen on my linux systems.