Sysadmin's favorite collection of infallible utilities failed … foully

Brace yourselves, gentle readers, for we have good news and bad news. The bad news is that the weekend is over, and you have to be back at work. The good news is that Monday brings an instalment of Who, Me? in which Reg readers entertain with tales of technical misfires. This week, the hero of our tale is "Duke" (not his real …

[GlenP]

Even Further Back...

defragging a disk could make a huge difference

A customer had two ACT Sirius machines, with whopping great 10MB HDs, to run Autodesk CAD. They weren't really our responsibility to support but they were complaining that saving files was becoming a huge issue, up to 20+ minutes for some designs, so I had a quick look.

Inevitably, with a small disk, low memory and an application that spewed out temporary files, the disks were so heavily fragmented it was a miracle anything ran at all. I ran defrag on both machines, ran a few more basic checks, advised the customer that removing a few older files and repeating the defrag monthly, or so, would help, and left a happy customer who could now save files in under a minute.

[Prst. V.Jeltz]

Re: Even Further Back...

In college we used to run autocad on 486 machines with no hardrives!

They pixie booted and everything lived in the 8mb of RAM .

[Marty McFly]

Re: Even Further Back...

"...PXE booted..." Preboot eXecution Environment.

Dang, where has that bit of trivia been lodged in my brain? I cannot believe that just popped back in from the cold storage archives!

[Michael H.F. Wilkinson]

Cleanliness can have its downsides

Reminds me of the reason I installed Windows NT 4.0 on our home machine. My wife tended to clean up things on disk, which is commendable in itself, but removing autoexec.bat, command.com, or config.sys can be unhelpful. After I installed NT 4.0 and didn't give her admin rights, things somehow ran much more smoothly. She was a bit ticked off at having to log in, but at least the thing would boot up properly.

[simonlb]

Re: Cleanliness can have its downsides

Similar one for me: When doing 2nd line support 20-odd years ago I had to work with someone who had somehow blagged their position presumably by outright lying and never having been asked any actual technical questions in her interview. One day she's trying to remotely sort out a client's PC running XP when she puts them on hold and asks everyone across our bank of desks, "Is it ok if I delete pagefile.sys on their machine? It's taking up a lot of space." Cue everyone stopping what they are doing and looking up at her, then looking at each other, speechless.

[yetanotheraoc]

Re: Cleanliness can have its downsides

At least she knew enough to ask.

[Mishak]

Reminds me of looking after my parents' machine

Dial-up internet access would fail a few times a year, necessitating a 360 mile round trip to fix it. Usual cause was dad changing settings "because".

Machine was already running NT4, and removing admin rights from his account meant it ran much better.

[Bebu]

defrag.exe no /FORCE:Y switch?

MS-Dos was normally such a pita with its switch syntax that I would have thought if defrag didn't detect at least something resembling a FAT file system it would have chickened out.

I have seen a system V/386 file system trashed when the PC was booted off a DOS floppy and said file system "fixed." Sorry fsck isn't now going return your data... all gone to the great archive in the sky.

Fortunately in those days almost all Unix machine ran on proprietary hardware (non intel x86) even if the CPUs (eg M680x0) weren't so this was a one off cockup and very sobering lesson for the (ir)responsible party.

[Lennart Sorensen]

Re: defrag.exe no /FORCE:Y switch?

Yeah I find it hard to believe too. If the disk doesn't look like FAT, doesn't a partition type of fat, I don't think any dos boot disk would even let you see the partition, never mind running any defrag on it.

[Tim 11]

Defragging isn't a big deal anymore

yes partly because disks are faster but mostly because 99% of PCs don't use disks nowadays

[tiggity]

Re: Defragging isn't a big deal anymore

@Tim 11

"yes partly because disks are faster but mostly because 99% of PCs don't use disks nowadays"

Plenty of us still running old machines (multiple in my case) with spinning rust disks rather than SSD.

It will be a while before SSD really do have 99% of the market of PCs that are still in use.

I'm sure someone can hunt some stats but I'm guessing even with current PCs that less than 99% ship with SSDs as OEMS do like to keep cheap / budget PC lines (keeping old designs and older technology chugging along) for the likes of PC World etc to flog to unsuspecting punters (most who will be blissfully aware that the "cost saving" they make buying such machines is only one metric and not factoring in the far better performance available for a slightly higher price)

[Peter Gathercole]

Re: Defragging isn't a big deal anymore

I don't know about you, but pretty much all of my systems, even the older ones, now have SSDs for at least their boot disk. And I'm even talking about some systems with IDE disks.

The only thing that is running spinning rust is my IBM pSeries (because SAS SSDs are still expensive), and my NAS box.

[Korev]

Re: Defragging isn't a big deal anymore

> The only thing that is running spinning rust is my IBM pSeries (because SAS SSDs are still expensive), and my NAS box.

The same here for the NAS, although it's getting to the point that I could afford to run SSDs now. If Synology made a NAS I wanted to buy (ie one that can do both 10Gb/s and video transcoding) then I'd probably take the plunge.

[Anonymous Coward]

Re: Defragging isn't a big deal anymore

If you like the OS then maybe a Xpenology self-build is the answer?

[Killfalcon]

Re: Defragging isn't a big deal anymore

My last spinning rust drive - either in home or office - is my aging PS4. But even so, 99% is a very large number and plenty of folk still use older machines, so I doubt it's quite that high.

It doesn't seem to be a thing that the Steam hardware survey covers, not sure if anyone else is tracking these numbers, though.

[nintendoeats]

Re: Defragging isn't a big deal anymore

Running an IDE machine off an SSD is a strange experience. A bizarre combination of lightning fast (cuz SSD) and still very slow (cuz it's a Pentium 1).

[John Brown (no body)]

Re: Defragging isn't a big deal anymore

"It will be a while before SSD really do have 99% of the market of PCs that are still in use."

The vast majority of PC users or corporate, and mostly using laptops these days, on an approx. 5 year replacement cycle. I'd not be surprised if the 99% figure is not all that out of whack. Probably most home users are on laptops or tablets too these days, so again, primarily will be using some form of solid state storage.

Those of us with home servers or even just small off the shelf NAS boxes with spinning rust are probably in the minority now.

On the other hand, most of those corporate laptop users are probably mostly running "cloud" apps much of the time, either locally in terms of their employer on-prem servers or remotely on AWS/Azure/Google and so may technically be using actual spinning disks :-)

[Peter Gathercole]

Re: Defragging isn't a big deal anymore

..and the SSD manufacturers tell you not to defrag. SSDs, because it can shorten their lives significantly.

[SVD_NL]

Re: Defragging isn't a big deal anymore

Luckily the tools we use have gotten a lot smarter too.

Windows doesn't call it defragmentation anymore, but disk optimization.

This includes defrag and trim tools for HDDs and SSDs respectively, and picks the one that is applicable.

Defragging would wear out SSDs very quickly due to the large amount of writes when moving data around, but using trim actually makes your SSD last longer!

It helps cleaning up unused pages, if that doesn't happen your SSD will keep moving around useless data during the garbage collection process.

(garbage collection is somewhat similar to defragging, it moves active pages into the same data block, making it less fragmented and more efficient).

So while defragging is no longer a big deal, do not exclude your SSDs from disk optimization because not using trim will shorten it's life too!

[Julian 8]

Re: Defragging isn't a big deal anymore

have had to defrag a disk recently as I wanted to add over provisioning and even though the disk had plenty of space, windows had decided to write at the end of the partiton so the inevetiable shrink command would not run.

Thank fully I had an old version of o&o defrag and license I could use.

Other than that, let trim and windows optimise do its job

[Anonymous Coward]

Re: Defragging isn't a big deal anymore

I wonder what you think the D stands for in SSD???

Solid State DISK

[Yet Another Anonymous coward]

Re: Defragging isn't a big deal anymore

I assume that when they are hyperdimensional-mobius-quantum storage devices we will still call them disks

[phuzz]

Re: Defragging isn't a big deal anymore

It's a disk, but it's not a disc

[Julian 8]

Re: Defragging isn't a big deal anymore

I still "tape" the programmes on the TV

[anonymousUser13]

Re: Defragging isn't a big deal anymore

Isn't is Solid State Drive?

[Keith Langmead]

Re: Defragging isn't a big deal anymore

Yep indeed.

Hard Disk DRIVE

Solid State DRIVE

[mirachu]

Re: Defragging isn't a big deal anymore

Solid State DRIVE.

[Prst. V.Jeltz]

boot mbr etc

I've always hated getting disks to boot, specially in win2k / xp /98 days

Some times they'll stop with some message like mbr not found , and then you're screwed , in spite of a perfectly good copy of windows on the disk its a reinstall , just to make the disk bootable again .

yes there were some incantations like "fixmbr" or other stuff , but making a disc bootable seems massivley over complicated and has only got worse .

It wasnt too bad pre windows when format /s would do it , but again - that involved a format .

This days with this UIEFI thing , the OS and firmware are horribly tied together in ways i'm not gonna try to start understanding . Too old and not in that role now.

I''ve a linux machine that nearly had to be binned becuase of some "secure boot" shit that I never asked for.

I uefi/bios upgrade on my home machine resulted in non booting c drive just last week for some goddam reason Bitlocker was mentioned in various warnings/errors .

Again , never asked for that to be installed , and it seems like it wasnt - or it was but hadnt actually encrypted anything as the data was clearly visible

JUST NOT BOOTABLE ! AGAIN!

cue windows reinstall and loss of license key. (I knew the key , but seesm microsoft didnt fancy honouring it anymore)

Also this whole process seemed to finish off a 2tb backup drive , which tbf had shown errors previously but after this debacle was not retrievable with or without data .

/rant

[Anonymous Coward]

Re: boot mbr etc

"Some times they'll stop with some message like mbr not found , and then you're screwed , in spite of a perfectly good copy of windows on the disk its a reinstall , just to make the disk bootable again ."

Or FDISK /mbr to rebuild it, from a bootable disk.

[SVD_NL]

Re: boot mbr etc

A clean windows install always comes with BitLocker device encryption enabled, but it needs to be activated (basically the suspended state that's also used when doing BIOS updates for example, this means the drive is encrypted but the key is not hidden in any way).

I imagine this could cause problems if that data gets corrupted or a pointer gets lost.

[Dave@Home]

Re: boot mbr etc

"A clean windows install always comes with BitLocker device encryption enabled"

Are you sure about that, I rebuilt this machine from scratch a month ago and it's off.

My laptop was rebuilt 6 months ago, again, off

[Jou (Mxyzptlk)]

Re: boot mbr etc

> A clean windows install always comes with BitLocker device encryption enabled, but it needs to be activated.

Nope, that behaviour depends on a specific set of flags in UEFI BIOS combined with TPM 1.2 or 2.0. HP machines are an example using that method, others manufacturers may do the same.

Even worse: Only using manage-bde.exe <driveletter> -off you can disable it for the drive without activating it first (see my answer in that thread).

Why I have to do this even if Bitlocker is desired: If Bitlocker is in that half state right after installation it does not use the highest possible encryption method available, it only uses AES128 instead of AES256-XTC in case of Windows 10/11. AND it may not store the recovery key in Active Directory since it is already beyond that step. Though the latter can be forced with manage-bde.exe after the encryption was already done.

[tip pc]

Re: boot mbr etc

longtime mac user, i learnt early days to have a partition for the OS & another for my data.

i could reinstall OS's etc to my hearts content relatively safe in the knowledge that my data was going to be safe.

Newer MAC os's do this by default now with immutable OS partitions etc to ensure integrity of OS on launch.

i remember Juniper did something similar a long time ago, 2 x boot volumes helped with OS upgrades especially if there was a failure on upgrade

https://www.juniper.net/documentation/us/en/software/junos/junos-install-upgrade/topics/topic-map/how-to-recover-junos-upgraded-freebsd-using-menus.html

[Prst. V.Jeltz]

Re: boot mbr etc

" have a partition for the OS & another for my data."

I certainly do that , not just a partition , a physical drive ideally. Usually pans out that way when you get round to buying ssd's for them to boot from anyway.

Its all come together since I've got a couple of NAS that back each other up , all my data is on those .

None of the computers in the house have any data on , so I'm not worried about them dyeing .

Bought 2tb and 4tb drives this morning to go into my main pc to serve as a third copy of the NAS data .

[trindflo]

partition for the OS & another for my data

I did that professionally in a few places. I have a windows installer that will just wipe the system partition and reinstall the desired Windows version leaving the data intact. It can be done, but required making my own 'unattend' script that launches other scripts. Powerful stuff, but not the faint of heart (or wallet).

[that one in the corner]

Re: boot mbr etc

> partition for the OS & another for my data.

And then Windows makes it harder and harder to not keep everything on the C: drive (not to mention an apparent decrease in installers not bothering to provide an option to choose the install path - you are really not helping, guys; and as for that little scrote who decided that Arduino should ever be less than 100% portable...)

[Lennart Sorensen]

Re: boot mbr etc

All format /s did was do format, then do sys. You could have just run sys c: and had it bootable without formatting it again. Fixing booting on dos based systems was really very easy.

[mirachu]

Re: boot mbr etc

Not "format sarcastically"?

[Prst. V.Jeltz]

You'd think defrag.exe would say "Hang on , this aint FAT , dont know what to do with it" rather than happily start munching away at a file system its not written for . What did it think it was doing ?

If you hired a German translator and it turns out you needed a French translator he's not gonna just start guessing and writing random English words - you'd get an error message.

[Killfalcon]

It's probably built to be fast, so it's going to do something like grabbing 4 bytes, treat the first three as an identifier and the fourth as a length, tag that length in bytes as that first identifier then jumping [length] bytes ahead, grabbing four bytes to get ID and lengths, jumping [length], etc, etc.

(I actually got myself into a similar mess a few years back, trying to write a world editor for Minecraft)

It's not until someone calls you up and says "your defragger ate my drive" you realise you've fucked up. Or maybe you always knew it only worked on FAT, asked the boss if there was time to add a test and they said to just put "only for FAT" in sharpie on the floppy disk and ship it.

[Doctor Syntax]

You'd think defrag.exe would say "Hang on , this aint FAT , dont know what to do with it"

Take a deep breath, stand back and realise this is Microsoft we're talking about. Microsoft with the Microsoft world view: "Everything is assumed to be ours.". Yes, NT was NTFS was also Microsoft's but defrag.exe was written before that existed. In the view of its authors, therefore, all disks were FAT and there was no point in even thinking they might not be, let alone testing for it.

[yetanotheraoc]

all your disk are belong to us

"all disks were FAT"

If they weren't, they are now.

In all seriousness, though, certain tools are meant for end users, and other tools are meant for those who are supposed to have RTFM. My first purchased PC (DOS 3.1) came with a printed manual that explained things like CHKDSK could be harmful. My most recent purchased PC (Windows 11) came with a picture of how to plug it in and where the on switch is. It's relatively easy to make a computer say "error", but aside from this Microsoft ships all the tools needed to make a bonfire of the installation.

[Anonymous Coward]

Re: all your disk are belong to us

It's a bit like the user handbook for a car from 50 years ago. The handbook explained how to set the timing, set the valve clearance and replace the spark plugs. Nowadays the handbook tells you not to drink the battery acid.

I believe it is called progress.

[Prst. V.Jeltz]

Re: all your disk are belong to us

bit of a popular misnomer / exaggeration really .

[phuzz]

Perhaps 'Duke' ran it with /NOPROMPT?

[Stuart Castle]

I remember when I first left school. Doing Tech support.. I always had my box of utility floppies, including dodgy copies of both Norton Utilities (when it for was good), Norton Commander, Xtree, Sidekick and Central Point software's PC tools, and whatever other tools I could cobble up. Using that little lot, I could usually have a good go at fixing whatever problems I encountered, and even though they were illegal, Viruses weren't really a thing at the time. When they did become a thing, I just added a copy of Dr Solomon's Antivirus.

That box has long since gone, to be replaced by an external SSD containing legit copies of the software I use to fix problems.. Usually freeware or open source, but in a few cases, I have purchased licences, or been given them by my employer. My SSD is in an external case (an iodd st400) that offers the cool facility to emulate CD/DVD/Blu Rays. I know that Windows, and pretty much every other modern OS, can mount ISOs directly, but because the emulation is done by the drive itself, and appears to the host system as a USB CD drive, it's useful for booting OS ISOs, as it works in the machine's firmware.

[John Brown (no body)]

"That box has long since gone, to be replaced by an external SSD containing legit copies of the software I use to fix problems.."

Next time you boot it up, go to device manager and then select View --> Show hidden devices. Prepare to be shocked! It might also explain why, sometimes, on booting, some devices you expect to work, no longer do. Too many conflicting drivers from the rich and diverse selection of PCs/Laptops it's been plugged into over time and Windows may get confused and choose the wrong one :-)

Just the other day I had to go in and delete the many dozens of "stale" camera drivers so the system could re-detect and install a "new" driver instead of trying to use the already installed one it thought was the correct one. I've not got the link to hand, but there's a freeware program out there that will delete all "unused" drivers from device manager. It saves a lot of time doing them one at a time because MS don't allow shift-click group selection in device manager. (I suppose this would be an edge case, and there may be good reasons to not allow someone to "accidentally" select a load of valid drivers and remove them all at once :-))

[Liam Proven]

I reckon someone somewhere misremembers this.

NTFS drives don't get drive letters under DOS and a DOS defragger wouldn't even run, let alone damage it. Ditto HPFS on OS/2.

It could have been Win95A or a later release with a FAT16 drive with long filenames. An old enough DOS version wouldn't recognise that and would try to defrag it, and would destroy all the LFNs, but it would still try to boot afterwards.

Or indeed an NT machine with FAT16, which was possible, but internally NT used 8.3 letter filenames, intentionally, to make it safe against LFN errors or damage.

[I don't know, stop asking me.]

I think I agree here.

The last time I defragged a disk, it was an MFM disk (or possibly RLL).

Since IDE took over (and then ATA etc.), there has hardly ever been a need to do so. So we're talking about very early 90s probably, firmly in the DOS / Windows 3.1 era.

[yetanotheraoc]

Good catch

Although with a bag full of repair options, if one doesn't work just try the next one. The article does say Windows 95 Office 97 era, so NTFS was around. Maybe it was the customer's mistake sending the _server_ to the usual PC repair shop, when it most probably had its own support contract. That would perhaps explain why they were so understanding when it was rendered unbootable.

[Jou (Mxyzptlk)]

As far as drive letters are concerned: I wouldn't be so sure that older DOS versions don't assign a drive letter to NTFS drives, and therefore see garbage. But I am too lazy to test now :D.

The real problem: We don't know what defrag tool they used, not necessarily the DOS supplied one. There may be some out there which simply try to defrag all partitions they see, no matter whether they have a drive letter or not.

[Anonymous Coward]

I did something similar

Windows 95 defrag seemed to be taking an age so I had the bright idea of running DOS defrag from a floppy.

Once completed I had a drive full of trunca~1.exe files…

Lesson learned. Alongside installing a 486 processor the wrong way, setting a PSU to 120v to see what it did, etc.

[AVR]

Clearing out c:/tmp (& similar) - you can't just do that! That's where some people store their most important files. There's the odd piece of software that gets annoyed when its temporary files vanish too, an old document management system I had to support springs to mind.

[John Brown (no body)]

On the other hand, Windows Disk Cleanup will do exactly that, only sparing files it knows are part of Windows and are currently needed. If the filename is not on the "safe" list, it gets deleted.

[John 110]

I used to defrag my Amiga HD regularly. Start it going and plonk the kids down in front (they were very young) of it to watch the little squares fly about. An easy way to get a quiet couple of hours to drink coffee.

[Turkey_Bender]

The old NT Manual Defrag

I seem to recall that at one time, the M$-approved method for defragging Windows NT was "Take a backup, format the disk, then restore the backup"

Ah, the good old days.

We had tech from a small-shop acquisition who was convinced that his department's slow performance was because we needed to defrag the NAS (which was an Appliance, on the Network). He even tried it once (fortunately the software he was using refused to run against network drives).

Turned out that the users located in the Midwestern US were configured to get AD authentication from London. But it took me 4 months to convince him to even look at that.

Don't miss those days.

[steviebuk]

It does mattee

Having dealt with PCs since the 90s and in my early days actually being careful with static I started to think it didn't really matter. I'd built a few PCs in my day on my bedroom carpet and all OK.

So in office back in about 2012 and get under desk of the IT Helpdesk lady and tell her I'll just shift the PC a bit. The cable will be loose on the graphics card. "Want me to turn it off first?". No I said, it will be fine. I wiggle it a bit while on carpet, having already rubbed myself on the carpet and fizz and pop and she says "Oh the PCs just turned off". Won't come back on.

Ah shit. I either shorted the card or static fucked it. I made up an excuse and replaced it with a temp :)