Boss broke servers with a careless bit of keyboarding, leaving techies to sort it out late on a Sunday Give thanks, dear reader – the weekend is upon us and it's a lovely long one for readers in the US of A. But that's not stopping us from bringing you another instalment of On-Call, The Register's weekly reader-contributed tale of the thankless chores IT pros are asked to perform. This week, meet "Dennis", who in the early …
Not really? Just requires some thought on where you put stuff. Put simply, if it's in sunlight it's hot, if it's in the shadows it's cold. If you need to keep your thing cold, put it behind something casting a shadow (can be as simple as a large mylar or other synthetic material whipple shield similar to the JWST sunshield, and reject heat into space using a radiator on the shadow side. Requires more thought than "a block of metal of fins with a fan" but still.
"can be as simple as a large mylar or other synthetic material " or more solar panels mounted with a small gap to the satellite.
I must have missed the memo that said solar panels must always be mounted well away from the body e.g. International Space Station.
If there is a reason for that I'm all ears - Dumbo
That and thermal management. What light isn't converted into electricity or reflected back into space is converted into heat, and that heat needs to be removed. In space, there is no convection or conduction to transfer/remove waste heat into the environment-- remember, space is vacuum, better than the gap between the inner and outer walls of your insulated flask/water bottle-- so you're left with radiation. Having the heat radiate into space is good, while having it radiate at the living quarters of the space station is bad-- they have enough problems getting rid of their own waste heat and don't need any from the solar array.
Haven't got a repurposed coat cupboard but here at home I do have a repurposed airing cupboard :-)
It used to contain the hot water tank for the old heating system, but that was removed when we installed a modern combi boiler. Which left plenty of space for router, printer, one of my modded Humax satellite boxes, a couple of Pi-holes - you get the idea.
And of course the holes between floors where the 15mm or 18mm pipes used to run are perfect for routing ethernet cables, plus the waste heat from the kit means it still works as an airing cupboard.
The Jupiter mission ship in the movie "2001 - A Space Odyssey" was originally designed with large surfaces for heat dispersion when the on-board nuclear reactor was supposed to be working, running the drive system. These were deleted because they looked too much like fins, which, of course, are for ships that fly in atmospheres.
Instead we had the "seminal" design that we saw in the movie, a spherical habitation module connected to the drive module by a long spine. The novel kept the "fins".
It depends. There's a subtle distinction there. Make dinner and then eat it? That's preparing. Make the upcoming week's dinners on the weekend? That's pre-preparing (especially if you don't eat the first one the same night you make them).
I'd never encountered "pre-preparing" before, but if the writer/speaker was using it in awareness to capture that distinction, I'd be fine with it. If they were just being sloppy, though, I'd absolutely agree with you.
Kind of like my bugbear: "instantiation", meaning an object. When you instantiate a class, what you get is an *instance*. "Instantiation" is the process of doing that, not the result. It's a perfectly reasonable word, just not with the meaning many writers ascribe to it
I live in dread of my first sighting of "instantiationize" -- i.e. the same error, repeated.
"It depends. There's a subtle distinction there. Make dinner and then eat it? That's preparing. Make the upcoming week's dinners on the weekend? That's pre-preparing (especially if you don't eat the first one the same night you make them)."
Unless you're preparing the preparations for the upcoming week's dinners (just putting out the required vegetables, meat etc) then you're still just preparing the upcoming weeks dinners. The first pre- in pre-preparing is nearly universally redundant and should just be left out. I cannot think of a single instance where adding it puts any different meaning on the sentence over just using "preparing".
The way I'd use the terms, it's only on prem if it's on your prem. If your admin can physically access the machine while working in your building, that's on prem. If it's in a colo somewhere, that's no longer your premises, with the difference that you're more likely to be accessing it remotely unless things are very messed up. Even before the cloud, having a machine that wasn't in a building you owned wasn't unheard of, and outsourcing email was similarly possible.
Having grown from the PFY to the BOFH, to the manager, I'm keenly aware that I can still help out a qualified engineer with crazy ideas or helping to carry a switch that is too heavy to carry alone, but unfortunately I also know not to rummage around anything all by myself, because I've grown into the single most scary thing to any engineer:
I have enough knowledge to do vast damage, but not enough knowledge any more to recover from the damage I caused myself...
I have enough knowledge to do vast damage, but not enough knowledge any more to recover from the damage I caused myself...
Something that I live by, as it's a good few years since I've been been properly on the tools
There are a few systems here which I sometimes have to refer to colleagues to get me information about - how many records are in such-and-such a table, what's the disk utilisation, etc. All stuff that readily accessible through a management console, but I explicitly said that I didn't want to even have a login for that.
For my colleagues the nuisance factor of me bugging them with the occasional question is far more preferable to me trying to answer the question for myself and accidentally doing some damage that causes them even more work.
Does dropping a largish card deck count?
I lost control of a handtruck loaded with punchcards containing a post-grad's entire dataset from an overnight run at SLAC. Down a flight of stairs (the elevator/lift was down for maintenance). I was a teenager, landed a position as a summer intern ... I thought for sure I would be out the door.
After giving me shit for about half an hour as we picked up the cards, Don (the post-grad) relented & demonstrated the proper care & feeding of a card sorter, and explained how columns 73-80 were used. And why there were lines drawn at various angles, in various places, in various colo(u)rs, across the edges of the cards in the decks in the "library". I'm pretty sure that's where I started developing my belt-and-suspenders (belt-and-braces to you Brits) approach to data retention. I'm not quite paranoid about it, but I'm getting there.
It was "rm -rf *" in what I thought was a sub-directory containing log files (organised into further sub-directories based on date, hence the recursive delete). Instead it was the directory containing the config files for the application.
In my defence, I had been warning manglement for ages that we were running out of disk space because my predecessor hadn't configured a purge policy for logs. It turned out to a good way to discover the config files in version control were out of sync with what was in production, and I reconciled the differences from one of the other application servers.
This post has been deleted by its author
Newbie.
On desktop systems, I started with cassette tapes, went to 8” floppies, then 5.25s, them 3.5s. Along the way I had a few mighty 5MB and 19 MB, and, dare I say it, 40MB hard drives. And some diskpacks, ranging up to the incredible, no way we could ever fill that up, 300MB.
Of course, there was the time that my home desktop system had a better CPU, more RAM and more storage than the Very Expensive supermini at work.
This post has been deleted by its author
"It was standard to first put Rouge on any new system, y'know, to test it."
I guess if one can put lipstick on a pig ...
"Ah, those heady days!"
That's HEDY!
Shame Miss Lamarr is remembered more for her paint & powder than her more technological inventions.
This post has been deleted by its author
On desktop systems, I started with cassette tapes, went to 8” floppies, then 5.25s, them 3.5s.
And don't forget. If you had a 3.5 and a drill, punched the hole at the right spot, you could double your capacity, just like that!
I also remember, when I moved from my vic20 to a 64, buying a floppy drive so I could move from cassette to 5.25. It took my earnings of many, MANY weekends (washing cars, filling shelves, distributing newspapers) to be able to buy the bloody thing. I never forget, it was a whopping £430/ € 500/ $520 (calculated from no longer existing currency). And that is not even taking into account inflation for the last... erm... 40 years. Still think about the thing when I'm rummaging on hardware sites and see floppy drives for < €10.
At $work, our production data archive safe is practically a museum of removable data storage. 5 1/4" floppies, 3 1/2" floppies, DC 600A tapes, LTO tapes, DAT tapes, magneto-optical, oh, and there's some of those newfangled CDs in there too. I've seriously considered taking some of the fresh-from-college engineers down there to teach them some history.
One of my colleagues was rummaging around in his desk and found a 3.5" floppy disk. Showed it to the helpdesk staff. They were fascinated. Pulling back the slider and poking at the disk. They were astonished when we told them it could store a massive 1.44MB as long as it was double sided.
The first machine I ever got to play with was a TRS80 Model 1 where my dad worked. It had 2 x 5 1/4 FDDs. I went from that to tape when I got a Spectrum at home.
"Along the way I had a few mighty 5MB and 19 MB, and, dare I say it, 40MB hard drives. And some diskpacks, ranging up to the incredible, no way we could ever fill that up, 300MB."
I've heard stories like this before, and I have to ask whether that was true. Not so much for a 300 MB disk, as I can see how that would look pretty large compared to files, but people who express similar sentiments with 5-20 MB drives. I know I'm demonstrating my relative youth, a serious blow to my standing in this community, but did anyone who got such a disk really think it would be hard to fill that? A novel consists of about a hundred thousand words, probably with an average length of about six letters. So storing just a text file of a novel would use up an eighth of a 5 MB disk. Even with a 20 MB disk, you could store 40 uncompressed books if we're being charitable. The fact that this disk stores less information than my bookshelf would probably have made me think I could fill it up if I wasn't careful to compress and prune data.
Yes, I know you weren't storing novels on the disks, or at least you compressed them first, but business documents and memos are also on paper and stored on bookshelves so the parallel is still direct. Maybe I'm just not understanding what you were storing those days.
"but did anyone who got such a disk really think it would be hard to fill that?"
It's mostly a matter of the difference in how computers were used, between then and now.
The personal computer class HDDs in the 5 to 10 meg range were mostly just to store programs that were used regularly, and documents that were a current work in progress. Everything else was stored on one of the myriad floppies that accumulated with every PC of the era. So yes, the concept of needing over 10 megs of online local storage[0] was close to non-existent. The user needed to be their own librarian to curate whatever files they deemed necessary.
With the advent of 40 meg drives, DOS could only see a 32 meg partition, so they were formatted into a 32 meg and an 8 meg drive. Programs and work in progress were kept on the 32 meg side, and data/finished work on the 8 meg side. Floppies were still used as near-line storage. Some of us learned that the "old" 10 meg drive could be used alongside the 40 meg drive, for extra storage. We were still librarians, with the help of such tools as Norton Commander or Xtree(gold) and the like.
By the time 80 meg drives became common, DOS could see the whole thing, so they were formatted as 80 meg drives ... and the rot started setting in. Program files and data co-existed on the single large partition, and the sight of a box of floppies next to each computer started to go away. The process accelerated with 120 meg drives becoming the norm. A year or so later, with 200 neg drives, floppies started to look like they were going to become obsolete. Norton Commander became almost mandatory for power users, because there was so much junk being kept in online storage.
And then Windows 3.0/3.1 hit, and kitchensinkware became the order of the day. Drives soon exceeded 500 megs ... and programs grew to fit the space. It's been a race between Marketing cramming more and more crap into programs and Engineering increasing the size of available disk ever since. Norton sold out, and even the disk utilities bloated beyond recognition.
Which brings us to today ... Windows 10 requires how much disk space, exactly? And if I need a word processor, spreadsheet and database it takes up how much more space? Just to create the exact same document, spreadsheet and database that I needed back in the mid 1980s ... Seriously, running a small business hasn't changed appreciably in that time, so why the program and file-size bloat?
I'll leave the answer to that question as an exercise for the reader.
[0] Back then, "online storage" meant "on the computer the user was sat at", as opposed to "near line" floppy storage ... off-line storage (which today the marketards call "cloud storage") pretty much didn't exist for most folks yet.
I wonder if Microsoft would allow me to whitewash their fence if I gave them my apple ...
I've heard stories like this before, and I have to ask whether that was true. Not so much for a 300 MB disk, as I can see how that would look pretty large compared to files, but people who express similar sentiments with 5-20 MB drives. I know I'm demonstrating my relative youth, a serious blow to my standing in this community, but did anyone who got such a disk really think it would be hard to fill that?
I'm going to answer this in a somewhat indirect way.
You've grown up with nothing but large HDD's and iPods etc and are thinking in terms of storing all of your data on it and nothing else. These days the only external storage you might have kicking around at home is DVD's for videos, but even then streaming is probably as common nowadays.
We grew up with zero HDD capacity and large libraries of 160kb 5.25" floppies, or 720kb 3.5" discs with the occasional double high density 3.5" floppy disk 1.44BM. This was quite natural since you did the same with cassette tapes for audio and VCR's for video.
Now I wasn't working, but I was doing something just as difficult in the day; gaming. This was not a "plug and play" experience, by any stretch of the imagination. I didn't start off with an HDD at all. You had everything on different floppy discs, some with applications, some with data and some with games. Getting a soundcard, joystick and mouse to work required cramming all their drivers on a single floppy bootdisc (occasionally along with the game) and then very precisely juggling the order in which they all loaded to into both the PC's base and extended memory to get everything to fit in the (very limited) available memory and still run.
We got very, very good at using every single last bit (if you'll forgive the pun) of space. For instance, many of us had custom edited the drivers and other software to only have the things in it that it really needed; for instance you didn't need the full OS, or the full drivers in many cases as they included the files for every possible configuration and you could ruthlessly prune these down to what is absolutely REQUIRED to fit a 720kb floppy and still run.
Now hand that same person with those space management habits a 40MB HDD. The space did appear limitless. And it was always there without needing to change floppies. Now we understood full well that you could fill that space; just shoving all of my files on it would probably have done it. However, our data had been living on removable storage for as long as anybody could remember and there was literally no reason to move it all to an HDD.
Given our extremely ruthless attitude towards wasted space (ie; habitually wiping out two thirds of an OS and drivers) i'd used about 25MB of my 40MB HDD in the several years before I was offered an upgrade to an 80MB HDD.
Just for general reference I declined that offer expressing an opinion that a larger disc was for people who couldn't learn how to manage their file space properly. :)
(Instead I think I was given a first generation 1x speed external CD ROM and some old data stayed on removable storage for literally years)
This post has been deleted by its author
Nope, but I've got a copy of the 12" floppy -- New Order's Blue Monday on vinyl, with Hipgnosis's brilliant sleeve design. (The floppyish holes and notches are actual cutouts in the outer sleeve, allowing the silvery inner sleeve to show through.)
It could hold the OS and WordStar and enough space to have a document or two.
Indeed. Or more PC oriented: the whole of WP4.2 on one floppy with room for your documents. Did what it had to do without distracting options popping up that somebody else thinks you are trying to do (Hi there, it looks like you are...). Sometimes wonder whether "Why are you moaning about that, because storage is cheap!" made all sloppy and killed those good house keeping principles (and gave room to those annoying tossers)...
Only because single-sided ones were cheaper. Then we'd use a disk notcher to lie to the computer that they were double sided.
(My phone just tried to autocorrect "double" to "fucked". That's surprisingly appropriate. I'd call it Freudian if the thing had a brain!)
Yeah, the server room is a strange place... cold in some places, like on the floor against the opposite wall the airco is mounted on. We used to put our lunch boxes there. One day a colleague brought in some goodies for a birthday party. They asked if they could put the bag in the server room. At this point in history, my office was adjacent to the server room, and had a door right from my office to the server room, which, back in the day was unlocked during the day. My colleague popped into the server room and deposited his bag of party snacks and was on his way. Later, I had to swap the backup tapes, and when I entered the server room, there was this odd smell of melted plastic. The shopping bag my colleague put in the server room, wasn't on floor where we all put our lunches, but behind a rack, so not to be seen by hungry eyes... It hadn't started burning or anything, but was a good approximation of shrink wrap on one side of the bag.
Needless to say, my boss at the time ruled out anyone other than server admins storing their lunch in the server room. I'm sure I should have had a good lecture from that, but I think if our colleague had asked my boss instead of me, my boss would have done the same thing, say yeah, and let our colleague proceed...
One place I consulted at had an "operator's room" from days of yore with a big window that looked out upon the datacenter. The operators were long gone, the room was now occupied by a few itinerant sysadmins who had to visit the datacenter regularly. Problem was, it was apparently pretty cold due to its proximity to the datacenter, even though it wasn't part of its climate control system.
So at some point in the distant past someone had cut a hole in the wall next to the window and put a louvered grill there, and run some flexible ductwork from it to the ceiling then down behind the giant fans blowing out of an IBM mainframe.
Since the datacenter was positive pressurized, losing a bit of hot air to an adjoining room wasn't an issue. If it got too warm they could damp or close the louvered grill.
I have no idea how someone had managed to get permission (or forgiveness) for having done this, or why it was allowed to remain, but it had been in place longer than any of the people working there could remember! Seems like it might be an issue for fire inspections, but maybe the inspectors aren't allowed on the DC floor or someone came up with a creative story to justify that on some sort of technology grounds the fire inspector wouldn't have the knowledge to dispute (since it disappears into the ceiling by the window and only appears some distance away at the mainframe, it isn't obvious at first glance they are related)
Hot and cold and creative.
Many years ago, i was working at a small electronic manufacturer / developer / do all shop.
One day we had an inspection from public safety and health officers.,...
They were not at all impressed when they found our climate chamber blasting at full polar condition to cool som crates of beer at the beginning of workday.
We had a lot of machines where alcohol and safety did not quite mix well (forklift, some pressfitting equipment, pritty toxic coating chemicals ect.)
But to be fair, the owner was a alcoholic sociopath who would fire people if they did not drink beer with him at his discretion - that kind of set the working environment, and types of employees.
But the icing on the cake was when they spottede frozen pizza slices dropping out of our reflow oven nicely placed between pcb's directly on the conveyor belt - Oh, and the production manager..on a few beers allready...dryly notede that the vapor phase, could do tinned soup.
Company stil in business, owner out, sons run it today. So AC. People that was there, know the name.
Spelling down to rusty English, and keyboard fighting i mothertounge, and hangover proberly
I heard about someone who wanted to move some disks - about 6 inches - while they were running; this was to make some space to get some more kit in.
They carefully put their backs against the disks, bent their legs (health and safety was important) and pushed. That worked fine until someone stood up. He caught his belt in the Emergency Power Off button - a "Pull" button, (to prevent accidents with people pushing it) and suddenly it went quiet.
This caused an immediate outage, and about half a day to recover. You cannot just power it on again.
Fortunately the guy's boss was one of those pushing the disks, and he got into trouble,
"laptops move around quite a bit"
The newer laptop I don't worry about because it has an SSD. The previous laptops I made sure to move them like molasses. Just turning them from the horizontal plane could have been hazardous. External drives I don't move them at all without spinning them down first, which means I have to think about it a bit before plugging them in. Even if 99.9% of the time it's fine to move them, in the not too long run they are dead.
Considering that current 2.5" hard drives from 20 years ago were able to handle 55 to 65G operating shock I don't think there is any possible concern about moving/carrying them while they are running. I never thought twice about it, I'd sling it under my arm carrying it at a 90* angle while it was crunching away on something then flip them quickly flat to set down and never had an issue.
You might survive 55 to 65Gs that would destroy an operating hard drive you were carrying, but if you live you will be so badly injured you won't give a shit about the condition of the drive!
The only way to apply that level of force is impact. You can't come close to 55Gs from any amount of flailing your arms about wildly while holding a laptop or tossing it in the air with a spin.
> 2.5" hard drives from 20 years ago were able to handle 55 to 65G operating shock
They could because they parked their heads away off the platter, not in the middle like they did before, as soon as the G-meter was either below 9.5 m/s² or above 10.2 m/s². A mild shackle made the drive park, no matter how important your job was. If you do that continuously you could get read-write-timeouts, but such situations are nonexistent outside of test stations.
The bad side: Some of those G-protected drives were too sensitive. If you typed too strong the heads could park while you work. You can hear the park-unpark sound in a silent office.
I remember a 5.25" 40MB MFM Seagate running off an RLL controller to give me 65MB. It was standing vertically on it's side outside the case for some reason I don't remember when I knock the desk with my knee. It fell flat and never worked again <sob>[*]
* Both tears and me shouting out Son Of a BITCH!!
Ah good not the only person, its a cupboard (So I would say no false floor and too small for the type of air con I would think about either freestanding or wall mounted) a leaning keyboard and the door closed caused the keyboard to be knocked over.
Maybe there was a small vent as it was a cupboard?
This post has been deleted by its author
One place I worked had what was called the "data hall". I think that was a hangover from when the previous occupants housed a bloody big old mainframe in there. We moved in and apart from some legacy Motorola "mini-computers" (remember that nomenclature?) all was 19" rack cabinets loaded with most windows NT4 servers and Cisco routers and switches (the "data hall" was became the main POP for our WAN). Because of the shape of the room, the legacy kit and the room's history the layout was a bit awkward. Basically the cabinets were located to match the a/c and power rather than the other was arouny. It wasn't the simple rows of cabinets you would normally expect in a DC.
When we moved it the room should really have been completely cleared out and rewired and the a/c redesigned. But management were (a) tight and (b) in a hurry, so it wasn't done right. They never understood that any job could be done any two of these three ways quick, cheap and right.
One of the legacies of this room was that along with the mainframe it had also housed big old line printers and tape drives which were considered to be heavy machinery and as such the mains feeds to that kit had a couple of big red emergency stop buttons wired in. One of these buttons sat by the entrance door a few inches above a couple of electrical dis boards. The dis boards were nicely flusht to the wall, but somebody had at some time decided they were ugly and needed to be hidden so some cladding was built aound them leaving a shelf a few inches deep below the emergency stop button.
You can probably see where this is going...
One day somebody brought in a delivery of some new servers. The double doors needed to be opened to get the delivery in. And once the trolley had been wheeled in the driver went to close the doors and bolt one half. At some point he decided he needed both hands for this task, so he put his clipboard down. You'd think he'd have put it on his trolley of server boxes. But nope. He put it on the "shelf" by the door. The shelf wasn't deep enough to lay it flat so he put it more or less upright, leaning on the wall. OK not the wall. Leaning on the emergency stop button. Having locked the door he turned and reached out for the clipboard. Caught his foot on his trolley, stumbled, reached out a hand to steady himself and yep. A lot of servers lost power.
Even though the room was protected by a big old UPS some of us decided we didn't trust that antique and when we moved in we brought our UPSs with us. One UPS per cabinet was only a few minutes of run time, but it was enough that it was longer than the time it took to find the key needed to reset the emergency stop button, insert it and turn it. Those teams who had decided to trust the room UPS weren't so lucky.
After that management were convinced that the room needed rewiring to suit it's current use. Properly done with dual feeds to each cab and no silly easily accessible stop buttons. But while that project was being scoped and planned and costed and all the rest that takes so long I carefully explained to management that this is why they should have had the work done before we moved in and I also made a couple of boxes to mount over the emergency stop buttons to prevent a repeat.
There has been much talk of these "Emergency stop" buttons in these pages .
WTF are they for ? I can see the benefit in a steel mill or a factory production line or a printing press , but a server room ?
Are they for stopping viruses ? what do i use instead of the Paris icon?
> to save people doing the 50Hz dance.
Yeahbut: why is a server more dangerous than a teapot or desk lamp with a skinned cord? We never red-buton these.
I have red-flap switches on my home heat burner (I see flames shooting out, I slap and run). And I have 'magnetic' red-flap switches which I plan to install on a table-saw so if power is lost it stays off, won't re-start with my thumb in the teeth. Neither sequence seems real likely on modern servers.
(My uncle probably knew a server fire, in days of line-printers and scads of chads, but he's long gone.) (Yes there was that OVHcloud server-to-building electrical fire last year but red-buttons may be the least of the factors.) (I see they were not the only ones.) (ElReg bites IT)
This post has been deleted by its author
I was told the big red button in our server room was to cut power in the event of a fire. Since the likely cause would be a short somewhere it made sense. We had a lot of geriatric kit in there (and a similarly ancient sysadmin) so the likelihood of component failures was high. Now all departed to the cloud (including our much missed sysadmin, albeit to a different kind of cloud in his case).
I was told the big red button in our server room was to cut power in the event of a fire
Correct.
Some years ago I had an interesting conversation with someone who designs (amongst other things) bespoke fire alarm systems for a living. At the time where I worked, we had a small server room - all our servers fitted in less than one rack, and we had three racks for cabling and switches, just a normal sort of SME setup. And we'd had a gas drench system installed.
So, the explanation I was given went something like this : in a fire you need three things, fuel, oxygen, and heat - the basic fire triangle that's on the first slide (OK, after the title side etc.) of every basic fire safety course. In our context, the heat comes from the electricity going in - most materials in our racks are self extinguishing, which means that if you remove the heat, they stop burning in a way that (e.g.) wood and paper don't.
So, remove the power going in and the fire will go out without any further action.
Now, what about these gas drench systems ? He went on to say that without them, as the equipment burns it will create chlorine free radicals which will immediately combine with oxygen (and other stuff) to form various relatively benign compounds. These gas systems inhibit that combination and leave the chlorine free radicals floating around. As you can imagine, these are highly reactive, and things like copper (e.g. the very thin and narrow traces on PCBs, connector pins, and so on) are especially favourable targets for their affections. So his suggestion was along the lines fo "yes, it will inhibit the fire, but all your kit will be useless afterwards anyway".
Hence the EPO button near the door - hit it as you're running away from the fire.
And you can consider having an automatic EPO linked to a fire detection system for when there's no-one to hit the manual EPO.
But, as this story highlights, you get nowt for free - and there's always the risk of an unintended EPO. So as with most things, you need to do a risk assessment and weigh up the pros and cons of all the options to decide which is the best compromise for your own situation. At one extreme, you have backups (should your data get damaged) and the risk of losing your one and only server room is higher than the risk of damage from an unplanned power off - so you use an EPO. At the other extreme, you're the command centre for a space mission and you really really don't want an unplanned EPO (in fact, they have battle shorts to over-ride circuit protection so a breaker tripping won't drop several bits of kit). And in between, you might have a fire detection system configured to trigger an orderly shutdown before dropping power.
Another downside of gas fire suppression is the sheer force of it going off. ISTR a story on this illustrious site of Glasgow City Councils data centre being severely damaged because the shock of the sudden pressure wave of the gas release kill all or most of their hard disks. I can't remember if this was a deliberate or accidental discharge.
Yes, that happened in our server room after a direct lightning hit to the building caused a massive surge & triggered the halon. The overpressure was enough to shift partition walls & bring down all the ceiling tiles, and lift some of the floor tiles. Many disk drives were found to be non-functional after everything was cleaned & restarted.
The only similar incident I remember was back in the 80s in Belfast, when a bomb blew in all the office windows and brought down the ceilings (fortunately after everyone had evacuated the building). At the time we all had VT100-type systems on the desks. Some had chunks missing due to flying glass, but despite the pressure wave every CRT was still intact and functioning. Almost every keyboard was dead.
There are a lot of people out there who are grateful for that 'Big Red Button'; their hearts continue to beat and there are others who have all/most of their digits, limbs, hair and other protuberances intact because of it.
After due consideration ---->
The most ... imaginative use I've ever seen: in one local outlet of a nationwide coffee-and-donuts franchise, the washrooms have power locks -- you have to ask a staff member to buzz you in. You buzz yourself out again using what looks like a genuine, data-centre-grade Big Red Button mounted just inside the door.
The first couple of times, I had to overcome some serious inhibition to actually push the thing.
I can just imagine the electrical contractor who wired the store spotting those buttons in a catalog and thinking of a brilliant off-label use for them.
This post has been deleted by its author
egress buttons are usually similar to the Big Red Button but are Big Green Buttons instead. Saw a set of 3 break glass UK fire alarm style switches in the DPU (data processing unit) (actually a server room) at work in the 80s... Red/Yellow/Green... Red for power shutdown, Yellow for Halon release, Green for 'i can't breathe open the door'
"egress buttons are usually similar to the Big Red Button but are Big Green Buttons instead."
Most of the time :-(
When they have a "push to exit" button and it's not the almost standard big green hemisphere, it can take a while to figure what to push to get out. I've turned off lights before because it's the only obvious nearby switch. In at least one case, the doors actually open so slowly that the "fix" was the place the button on the wall about 10 metres down the corridor. (An old Victorian building with heavy solid wood doors) The "fix" to the button being so far away was to place two signs. One near the button on a "music stand" affair that's actually quite hard to miss telling you this is "The Button" and another at the door telling you to go back 10m to find the button. :-)
Back in the day, San Diego State College had an IBM 1620 Mod. I that was run by student operators. The IBM CE gave the ops a simple rule to follow: If there is smoke coming out, use the normal power switch (it ran a cooldown sequence). If there were flames coming out, pull the Emergency Power switch, since the system was probably going to be a pile of scrap at that point anyway.
I almost had to use one once - when our brand-new IBM S70A started to produce copious smoke out of its vents and flames started to appear. However I first yanked the 3-phase power connector out the back instead, thus saving our farm of AS400s and notes servers from an sad end. No sparks (fortunately) but the S70A was sent home and replaced after being squirted with the CO2 extinguisher ...
"They never understood that any job could be done any two of these three ways quick, cheap and right."
I bet they did understand that, they just always chose quick and cheap. From their point of view it makes sense, all the more so since, being in charge, later when it goes south they get to decide who gets the blame that it wasn't done right.
I'm curious about why the UPSes were on the server side of the Big Red Button. Seems to defeat the purpose of an "instant" emergency shutdown, any equipment fire would continue until the UPS batteries were exhausted. When I set up my own home server cupboard I had an external BRB which cut the circuits on both sides of the UPS.
In the beginning of time, the reset button was exposed.
So everybody kicked it by accident.
Then it was recessed, behind glass panels, under key.
Then it was declared not good, cumbersome, unnecessary safety, and exposed again.
Then my cat, last Sunday, managed to sneak into my office, climb on my printer, then on my desktop, and hit the reset button with her little claws, while I was gaming.
I have no idea how am I going to cover that reset button.
"I have no idea how am I going to cover that reset button."
Duct tape and cardboard, of course.
In all seriousness, sometimes the "switch" is just a plastic plunger that makes contact with the actual switch further into the case. In this case, remove the plunger and keep a pencil nearby to hit the actual switch with. I've cat-proofed several servers this way.
Boss decided it was high time to test the aircon. In the middle of a very busy day.
In the building we had recently moved in to, aircon was provided by 2 VERY large, belt driven fans in a separate room. The control was - a big red button. He pushed the button, the aircon shut down. All good so far. Except he couldn't work out how to get it restarted. We had to call out the aircon engineer - who then rotated an almost invisible silver collar which had locked the button in the 'off' position.
Fun times, fun times indeed!
Ah, war stories - always happenz during weekend 3am uplifts, which usually fail - and we techiez come in riding a white horse to save the company!!
All done before Monday Morning Market Open and/or Black Friday - whichever comes first...
;-]
Remember, GENZers - Old mainframerz never die; they just fade away... into quantum computing worlds to come...
;-]]
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
