back to article I built a shed once. How hard can a data centre be?

The world of IT and hard physical labour are unlikely bedfellows, but a troublesome VAX installation brought the two together in today's On Call. Our tale comes from a reader Regomised as "Fred" who was a field engineer back in the 1980s. "I had a customer," he told us, "who informed me they were going to purchase a two- …

  1. Steve Davies 3 Silver badge
    Thumb Up

    Thanks for the memories

    As I read the article I had a big feeling of 'been there, done that'.

    My experience was with a pair of AlphaServer 4000's in farthest [redacted]stan. Six burly men carried the things up four flights of stairs to the DC of the Bank. Then they did the same to a 900lpm Line printer (remember them?)

    Aircon consisted of opening the windows a bit more or less.

    Those were also the days of X.25. The bank talked to Visa and Mastercard via X.25 that went over a Satellite connection. It was fun getting it all to work.

    Those were the days. Hinder & Stop were 3000 miles away and 25 years ago.

    Thanks for a lovely intro to 2021.

    1. StargateSg7

      Re: Thanks for the memories

      OMG! I so remember stuff like this back in my Oil & Gas Boomtown Days in Calgary in the late 1980's/Early 1990's --- We had numerous VAX 8000 series models and we bought one of the first sets of VAX 9000 supercomputers with the vector processor in the world in late 1990 at the time doing reservoir modeling!

      They were and STILL ARE total beasts! IN terms of size, weight and power consumption, they are HUGE!

      Our company owner at the time though was ONE of the smart ones who not only bought a boatload of experts from DEC (Digital Equipment Corporation), IBM and Cray PLUS the local provincial power company to setup the power distribution system and server room itself on a raised floor in a water-sealed, water cooled, air conditioned, multiple redundant power supply system with THREE noise-sealed 3 megawatt backup generators in a VERY secretive walled off part of the SUB-BASEMENT parking lot floors of the Petro Canada Tower in Calgary, Alberta, Canada.

      All you Calgary oil and gas workers never knew you were walking past one of the LARGEST data centres IN THE WORLD at the time! Five VAX-9000's, two Cray Y-MP's and four IBM ES-3090 series mainframes plus MANY Sun workstations and SPARC servers, Compaq '386 UNIX servers, IBM Model 80's and other branded RISC workstations all doing Oil & Gas reservoir modeling and searching!

      The power bill I absolutely KNOW was astronomical per month AND add in the FIRSTin-World setup costs of tied-together ISDN and Satellite lines giving aggregated data transfer rates at many Hundreds of Megabits per second at the time which was much more that what anyone ELSE had at the time!

      I remember lifting those MicroVAX box on wheels which weighed at least 150 lbs (70 KG) when filled up with memory and disks. The VAX-9000's must be around 2000 lbs+ although i'm not sure what the final weight is BUT we do still have the VAX 9000's in WORKING CONDITION as of today in 2021 doing 56k to 128-kbit modem-based non-real time financial systems work where their slowness BUT 24/7/365 data crunching consistency is an advantage! The VAXes have paid their multi-million dollar price tags MANY times over that last 30 years and are STILL WORKING whereas those multiple $40 million dollar Cray Y-MPs were recycled/scrapped in 1998 because of the cryogenic cooling issues damaging so many sub-components over the years!

      I do must admit I really miss DEC and their bulletproof machines. Their computers STILL do the world's mundane technical and data processing work even today 30 YEARS after they were first built. That VMS OS is ALSO bulletproof! They can and STILL DO have THOUSANDS of simultaneous users/tasks running tiny financial batch jobs 24/7/365 that make a few hundreds of dollars each day in low-level trades that when aggregated make up TENS OF MILLIONS of dollars worth of profit for the shareholders! Sometimes the OLD technology is STILL the BEST technology!

      V

  2. Blackjack Silver badge

    Always nice to have huge windows on the room upstairs you are moving things to.

    Then you can lift the not piano to whatever floor it is.

    1. Stoneshop

      Huge windows

      allow a fine view of the Director's Jaguar which has just gotten flattened quite a bit by a computer system that crashed. On top of it, caused by the need to manhandle the system that was to be installed, on the third floor, the lift not being large and capable enough, and one of the Burly Men losing his footing halfway up the second set of stairs.

      Although it was more like a huge hole where the huge window had been.

    2. John Brown (no body) Silver badge
      Joke

      You should always make sure you have the right equipment for the job too.

    3. Anonymous Coward
      Anonymous Coward

      Agreed that windows a good alternative

      I joined a bank that already had VAX 750s installed.

      We replaced with smaller and way more power efficient MicroVAX 3100s.

      The DC was on the 2nd floor of a Victorian building, with a decent staircase for normal use.

      Having had a team lug a large line printer up a few months earlier, it was decided to crane the old VAXen out via a sash window (with sashes removed). Dry but windy weather made it interesting for the crane driver. Luckily all went to plan for a change.

      1. Strahd Ivarius Silver badge
        Linux

        Re: Agreed that windows a good alternative

        Linux would be of course even better than Windows (first time I see Windows, whatever the flavor, deemed a "good alternative" )

  3. Anonymous Coward
    Anonymous Coward

    I got paid for xmas overtime.

    My employer at the time shall remain nameless to keep the laughter from crushing their itty bitty egos again.

    Our data center was located in a co-location facility along with multiple other companies.

    It shared power and aircon and water, etc, all on multiple redundant supplies, that way if one failed it *hopefully* wouldn't take down the entire center.

    (I bet you can tell where THIS train wreck is going, huh?)

    Xmas eve rolls around and we're all told to enjoy the holiday with our families.

    I was the low man on the IT totem pole and given the emergency pager.

    Had I known then what I know now I never would have accepted the damned thing.

    I've just sat down to xmas supper with my family when the bloody pager goes off.

    I excuse myself, call in to ask WTF, and get told that our data center has gone offline.

    "Get there ten minutes ago and get us back online an hour ago!" my boss yells at me down the line.

    I wisely refrained from mentioning that time travel wasn't in my job duties and I'd charge extra if they were.

    I hang up, give my folks the bad news, and head in to work.

    Where I find out that the aircon had caught fire, the firemen were on site putting it out with chemicals (something about multimillion dollar computers not liking water for some odd reason) and told me I'd have to wait to enter the colo.

    Fine by me, I'm getting paid by the hour and holiday emergency rates at that.

    It took them nearly two hours before they declared the site safe to enter, by which point the pager has gone off half a zillion times.

    Each time it does I have to trek over to a local phone booth (remember those? Cell phones weren't a thing yet ya young whippersnappers!) to call in and explain why I hadn't fixed it yet.

    Every time I patiently explain that the building had caught fire, blah blah blah, and I'd get on it just as soon as they deemed the building safe.

    Boss is screaming we're losing thousands of monies every minute and I need to get us back up NOW!

    Repeat that bit about the fire department & cheerily hang up.

    Anyway, I get in to find the colo *packed* with all the OTHER techs from all the other companies whom have sent out their own troubleshooters to the scene.

    Turns out we were ALL getting holiday emergency OT rates for standing around watching guys playing with their hoses.

    *Snerk*

    There isn't enough room for all of us to be in the room at the same time, so the senior techs go first while us lowly PFY's have to wait in the hall.

    Fine by me, I'm getting paid by the hour.

    About an hour and a half, maybe two hours later it's finally my turn to enter.

    The water isn't the problem; the power isn't the problem; the lack of any functioning aircon however is a BIG problem.

    And not one I can DO a damned thing about since I'm not part of Building Maintenence and *they* are the only ones allowed to futz around with internal building subsystems.

    They have been called out to try & get the aircon back up and running, but meanwhile I'm stuck there deciding which thumb to sit and spin on in my boredom.

    All the other techs have verified that the racks have power, the UPS' kicked in long enough to ensure proper shut down, and the physical hardware is peachy.

    I am fairly (nearly 99%) confident that our equipment will be in a similar state, needing only a reboot and a bit of hand holding to make sure it comes back up in a fit state.

    But I can't DO that until the building guys arrive... and I'm getting paid by the hour.

    Pager buzzes, find a working phone, call in, explain AGAIN about the situation, remind the boss that I've only had access to the computers for the last few minutes, and NO I'm not able to restart any of it until the building folks get that aircon up & running.

    "No aircon in a room full of heat generating servers makes for a fatally hot room. NOBODY is allowed to run their machines for any longer than it takes to do a POST check of the basic hardware. We are NOT allowed to leave them on as they will generate too much heat. Heat that the room can't get rid of without a functioning aircon unit."

    Boss snarls fine, tells me to stay there and get it running ASAFP.

    (Chorus)Fine by me, I'm getting paid by the hour.(/Chorus)

    The building guys show up about an hour later, hit the roof to examine the wreckage, and deliver the "good news / bad news" to us techies waiting in the colo lounge.

    The original aircon unit is a total loss and can't be repaired in any reasonable amount of time.

    The good news is that they had a replacement already installed waiting only for a thorough inspection before they turned it on and made it take over.

    "We'll have it back up in about an hour. Is that alright with you?"

    Hell yes. Why? Everybody sing it with me... (Chorus).

    Once the building guys had the new one online and told us techs we could bring our computers online, the senior techs got to restart theirs first while us PFY's had to wait.

    (Chorus).

    I finally get my turn, bring the servers back up, hold their hands until they're humming right along, and turn their network connections to live.

    I'm sitting there waiting to make sure it's not going to fall over when the pager goes off.

    I find a landline, call in, and my boss tells me "It's working now. Go clock out before I fire you for incompetence!"

    I got signed statements from the other techs about how long I had to wait before I could even start *and why*. I got signed statements from the firemen as to why it took so long to clear the building before they would even let me in. I got signed statements from the building guys why it took so long to resume aircon. And I took it all home with me since it was less than an hour before my normal shift started.

    I went home, changed, showered, grabbed some fast food for breakfast on my way in, and slapped said paper statements on the copier as soon as I made it to the Xerox machine.

    Copies went into my desk, the H.R. manager's inbox, and more handed to my boss.

    He asked what they were for.

    "To make sure that I get paid for all the OT I've spent, the fact that you told me to clock out less than an hour before THIS shift started, and that if you DO try to fire for ''incompetence'' I'll have something to defend myself with in court. Oh, and by the way, I've legally showed up for my current shift after NOT getting eight full hours since my last one. That makes *ALL* of the time I spent at the colo as OT to start with, the holiday emergency rates will apply *on top*. You owe me for 4 hours at a minimum and I'm going home to sleep now. See you tomorrow!"

    I've never slept so good in my life.

    I show up to work the next day and get met at the door by H.R. who is NOT happy to see me.

    Evidently the Legal department had been brought in on it and explained in NO uncertain terms was ANY form of retaliation to be done against me left I "sue us into a shithole so deep we'd never see daylight again."

    They had to get Upper Management Permission to cut me the cheque to pay me for my work that day/night/day, and Uppper Management was currently chewing the boss a new ass through which to shit from.

    Budget? He was now seriously over his.

    I left a few months later when one of the techs from the scene called me and asked f I wanted a job. His company was hiring and he liked what he'd seen of my skills.

    I said yes, put in my two weeks notice, and damn-near glided out the door on Mercury-winged shoes of joy.

    (Singing merrily)

    Tis the season to be jolly,

    I got paid... by the hour!

    Tis the season to go shopping,

    Because I got paid... by the hour!

    Now I'll don my gay apparell,

    Ha ha ha, ha ha ha, HA! HA! HA!

    I put a down payment on a new car,

    Because I got paid... By. The. Hour!

    (/Singing)

    1. GloomyTrousers

      Re: I got paid for xmas overtime.

      Thanks, that was a great story and worth an On Call article by itself!

      Happy New Year, all.

      1. anothercynic Silver badge

        Re: I got paid for xmas overtime.

        I second that. That. Was. Glorious.

        When you work for a dick of a boss like that, you were damn right to get as much proof of it not being your incompetence. Well done.

    2. Paul Herber Silver badge

      Re: I got paid for xmas overtime.

      An IT episode worthy of Ripping Yarns.

    3. Stoneshop
      Flame

      Re: I got paid for xmas overtime.

      Fire accounted for a nice bit of overtime pay once, although there was no hassle at all getting it paid out.

      At the end of one workday a colleague and I head to the train station 1.5km away, where we notice a huge mass of apparently-not-travelling travellers. Still made our way to the central hall, where the Tannoy was repeating an automated message that there was no current departure info to announce on the signage and to look at the departure sheets on the platforms and in the hall. "Doesn't look as if anyone is going anywhere anytime soon, all the tracks are utterly devoid of trains. So clearly there's Big Shit happening.". I call a colleague still in the office, tell him what's happening and to forget about going home for at least a couple of hours. A little later I get a hint that there's a fire in the train control room. Which can be seen from one end of some of the platforms, so I go there to check. And indeed, there are wisps of smoke coming from some of the ventilation shafts and one open window on the utilities floor. Then the platform gets cleared out by police and safety personnel. We head back to the office, where we get an update: the control room had to be evacuated due to smoke from smouldering cable insulation getting into the ventilation system, and without humans at the controls, even with automation running 95% of all required actions trains had to stop. Some even midway between stations.

      Yet, all the computer systems are still running so there's nothing we need to get involved with. For now.

      Then at around nine in the evening EVERYTHING goes dead. Outside power feed AND the UPS are cut as the fire brigade had concluded they were getting nowhere with foam and powder, and they were going to resort to water. Lots. Very, very lots. And no moving electrons interfering with that, so *klonk* go a couple of Big Red Switches.

      In the meantime I had been asked to go there again to get a more detailed status and see if there was anything we needed to do once the smouldering was stopped. Greeted by another colleague who was inside the moment the power was cut, and unpleasantly surprised by it.

      Sit around, consult with Energy & Lighting on how to proceed once power is restored, work out the sequence of what to start first, then wait until we're allowed access, around 0:30. In the meantime wo colleagues have offered to relieve the colleague at the office and me, but we consider it more prudent that they hit the sack *now* and take over early in the morning; it's impossible for us to go home anyway.

      The computer rooms are fine except for one door that has been subjected to an Obelix Knock, and one by one I start our systems and their peripherals. People come in to the control room, run their checks on being able to control signals and points, start collecting the location of the stranded trains, and get one to come in to the station. Job done, I head back to the office, report the status so far, have a taxi come to take me home.

      Over the next couple of weeks there were some more busy nights to work with a specialist cleaning company who went through all the systems cleaning their insides.

    4. rnturn

      Data Center Down

      I didn't get a page. I got a phone call from my manager on a Saturday morning. Thinking I'd screwed up and slept through a page, my boss assured me that it wasn't one the clusters I was in charge of but that the entire damned data center is down. It turns out that, after ignoring the pleas from the technician who'd previously alerted management to the problem, the UPS supplying the data center finally fried after running so close to maximum capacity for an extended amount of time. So the company experiences a lengthy /unscheduled/ down time -- during year-end processing -- rather than the much shorter one the tech had repeatedly requested. Major components of the UPS were going to have to be replaced. I got into the data center along with a dozen other admins and we all physically switched off the systems while the work on the UPS was taking place. Oh... the UPS? Some Einstein installed it so close to a wall that it couldn't be worked on---the vendor had to cut through the drywall in the adjacent room in order to access the fried electronics. Oh yes, the UPS vendor also determined that whoever had installed the transfer switch did that incorrectly, too---the backup generator was, essentially, useless and correcting that was going to be needed as well. When the work was all complete, management was anxious to restart all the systems. Until I asked: "So the repaired UPS has successfully been load tested and is good to go?" "Well, No..." "Don't you have a dummy load to test the UPS with?" Awkward silence "So the plan is to test the repairs using the production systems as test loads?" After a lot of hemming and hawing, someone from the UPS vendor finally admits that they /do/ have a dummy load... but didn't think to bring it with them. More delays while that's brought onsite and the UPS tested. Roughly a day and a half later, we're back up and running, full of free coffee, donuts, and pizza, and on a first name basis with the electrical contractors.

    5. Anonymous Coward
      Joke

      "he liked what he'd seen of my skills."

      Those skills in answering the boss, waiting your turn, and turning on some computers?

      1. The commentard formerly known as Mister_C Silver badge

        Re: "he liked what he'd seen of my skills."

        I would hire based on AC's careful and comprehensive gathering of CYA paperwork. Obviously an experienced management wrangler and therefore an asset to the team...

    6. Outski
      Pint

      Re: I got paid for xmas overtime.

      Gloriously written :o)

    7. Umbracorn

      Re: I got paid for xmas overtime.

      This really does deserve to be it's own On-Call, with flavour and writing flair preserved.

  4. don't you hate it when you lose your account

    Sounds like my house

    The stairs will only accommodate a double bed, had to take them apart and then rebuild them to get a queen size to the bedroom. Tech angle? Lying on it while I type.:)

    1. DJV Silver badge

      Tech angle? Lying on it while I type.

      Ah, the best tech "angle" - 90 degrees from the vertical!

      1. Marcelo Rodrigues
        Paris Hilton

        Re: Tech angle? Lying on it while I type.

        "Ah, the best tech "angle" - 90 degrees from the vertical!"

        180 degrees has its merits too...

    2. Boo Radley

      Re: Sounds like my house

      In college we were given a PDP 11/70 along with various tape and disk drives, that were all being thrown out. Geeks that we were, we took it all home, where we had to disassemble every component from every rack, and trudge it all up three narrow staircases to the fourth floor attic. The racks themselves were hoisted up and taken in through the thankfully big windows. Last up were four huge window aircon units, the biggest we could find.

      A resister placed in the proper location in the power company's meter kept our monthly electricity bill down to $50. We had multiple VT100 terminals all over the house, and mostly used the system to play DND. Those were truly fun times!

      1. rnturn

        Re: Sounds like my house

        Hmm... Normally, 11/70s needed three-phase power.

        If it were me, I'd have installed it on a lower floor and let it heat the house. When I last encountered one of those systems, they had a thermal switch in the CPU rack that would power off the system when the air conditioning went on the fritz---otherwise, it would have cooked itself to death.

        1. Stoneshop

          Re: Sounds like my house

          Hmm... Normally, 11/70s needed three-phase power.

          Not necessarily. Depending on configuration they could draw more than a single circuit can handle, and the prudent solution is to feed it 3 phase but the power supplies feeding the logic don't care at all if they're fed from different phases or not. However, several disk drives of that era would require 3 phase; I've heard of at least one case where the phases were wired wrong at the feed to the local breaker panel causing the disk drives to spin the wrong way.

          Totally the inverse of this is the Alphaserver DS20. It has three separate power supplies, needs 2 to start up (and it has 3 for that n+1 redundancy), but they MUST be on the same phase or they'll crap out.

        2. J.G.Harston Silver badge

          Re: Sounds like my house

          With the right equipment you can bodge 3-phase supply from single phase. Was one of our projects in C&G ElecEng.

      2. SImon Hobson Bronze badge

        Re: Sounds like my house

        Ah, memories.

        When I was at uni, the computer club met in the physics lab, and one week I noticed a DEC thing sat on a stair landing with a notice on it "free to good home". Not long afterwards, I was re-introduced to it in a friend's room - yep, he'd grabbed it. I think that was a PDP-8.

        The story i was told was that this had been declared redundant many years before, but one particular lecturer refused to let it go as is was so good for teaching computing fundamentals - the ability to toggle switches, single step it, and see what was going on by watching the lights. This lecturer was on holiday for a couple of weeks, so the rest fo the department took the opportunity to get rid of it while he wasn't there !

        So whenever I visited that friend, we'd always end up sat rewinding rolls of tape while we talked !

        I still remember some of the details. The processor alone was about 3feet of 19" rack, it had not just punched tape - but .... drum roll ... hard disks, two of them. I believe each disk held something like 32k words of data. And as well as the slow punch and reader on the floor standing teletype (ASR33 ?), this beast had both high speed punch and reader - so it could create chad far more effectively. Two full height 19" racks, and just enough room to still get his bed in.

        1. Stoneshop

          Re: Sounds like my house

          I still remember some of the details. The processor alone was about 3feet of 19" rack

          Processor cage plus an OmniBus expansion cage then, most probably. We got one from the University of Wageningen, one of the agri departments. Two low-ish racks, 3ft and 4ft high, an ASR33 and a panel with several dozen BNC connectors hooked to A/D and D/A interface boards, clock signals and some digital I/O. Mounted on a cart, it had been used to monitor environmental conditions for plant growth experiments in a greenhouse. Its program was still in core after having been put aside at least 15 years earlier, and would log to the ASR puncher.

          Two full height 19" racks, and just enough room to still get his bed in.

          In University I raised my bed to sit over a cupboard and built a workbench in the remaining space underneath. Setups like that were fairly common, and I can easily picture a bed on top of 19" racks; a couple of years ago I've heard of someone with an 11/785 in his apartment; with his bed over it.

      3. Old Used Programmer

        Re: Sounds like my house

        When I was in college, the EECS computer club was given a Univac SS-90, complete with card reader/punch and line printer. I drove the peripherals from Ft. Bragg, CA to Berkeley, CA using a 6-ton stake bed truck. My passenger was more than somewhat alarmed when I made it from Ft. Bragg to Willits in an hour. (PDP-11s were still in the future.)

    3. Dave314159ggggdffsdds Silver badge

      Re: Sounds like my house

      You took apart the stairs, not the bed?!

    4. David Hicklin Bronze badge

      Re: Sounds like my house

      All furniture going up our stairs has to be disassembled.

      Looking hard at the 3.4m long hand rail we want to replace and wondering if that can be got out in one piece as the replacement will be the same length, or if the the house was built around it.

      Doing a test run in the next few days !

      1. Strahd Ivarius Silver badge
        Holmes

        Re: Sounds like my house

        You should do it the old holistic way:

        Create a modelling software to evaluate the possibility of moving the furniture up the stairway (especially if it is L-shaped).

        Then you can fail to explain how the couch got stuck in the middle.

  5. Doctor Syntax Silver badge

    I've mentioned before the time, about 50 years ago, when a scintillation counter was delivered to QUB. About the same height as a typical rack but half as wide again and largely occupied by a complex sample changing mechanism. It arrived in a van without a tail-gate lift. It must have been loaded with a fork-lift but nobody advised we'd need one to unload it. It was eventually unboxed in the van, lowered onto its side and slid down a long plank. I don't think we'd worked out whether we could get it through this door and the narrow passage way behind it but it did just fit. https://www.google.co.uk/maps/@54.5858855,-5.9347162,3a,75y,165.71h,80.43t/data=!3m6!1e1!3m4!1st90JWAEKBeh2W1l4YPAH_w!2e0!7i16384!8i8192?hl=en

    1. Solviva

      Evidently some folks in that street are unable to read signs on walls...

      1. Ian Johnston Silver badge

        Or they're just exorcaising their raights to use the Quane's hayway.

      2. BenDwire Silver badge
        Facepalm

        That car is an Audi - what did you expect?

      3. Alan Brown Silver badge

        fire brigade inspectors are the solution to that particular problem.

        They won't touch the car on private property, but the H&S fines dished out to building owners will ensure bollards are placed to prevent repeats - and I'd like to see a driver ignore those

    2. Gene Cash Silver badge

      And now I know what "mews" means. I learned something new this year! I should quit while I'm ahead.

      1. DJV Silver badge

        'And now I know what "mews" means'

        Never owned a cat then! :D

        1. Stoneshop
          Angel

          Re: 'And now I know what "mews" means'

          Mewing loudly and pretending not to have been fed (which was done already by the other servant) is called 'fake mews' around here.

    3. G.Y.

      ropes

      The Hebrew University got a PDP11/70 delivered, late on a Friday, on a pickup truck. They told us "600Kg, 6 packages". Turned out one package (the 11/70) was 500Kg.

      We (the CS department) got it off the truck by tying ropes from it to a higher floor, letting the truck move, and putting a table to stop if moving (said table became matchwood). Let it there until Sunday, when the staff came in.

      11/70 worked like a charm

    4. davenewman

      Was that before the School of Management moved in or was it further along the Mews?

  6. Stoneshop

    Not encountered a situation as extreme as this

    But cramped computer rooms, aircon and power problems, sure.

    One site that ran on just half its cooling capacity as the other half had a pinhole leak and they'd already used up their Freon quotum so it couldn't be refilled. To beef up the cooling capacity they had installed lawn sprinklers underneath the outside heat exchanger, and on warmer days they simply opened a double door to the outside with a couple of fans in the opening. It didn't help that this was in a 'temporary'[0] building, wooden construction, tar-paper roof. They could have saved quite a bit of water, replacement disks and overtime pay (someone had to turn off the tap once outside temperatures had dropped enough; they hadn't even thought of using one of those sprinkler timers) by dropping a bucket of white paint on the part of the roof over the computer room, or putting a few rolls of aluminised bubble foil over it

    Another was experiencing network problems, which I found were caused by bits of underfloor 10b5 Ethernet gear being engulfed by a custard-like substance, the unholy alliance of linoleum flooring and glycol-based coolant.

    Although those were rather mundane compared to what tended to be posted in NOTED::WAR_STORIES. Oh, to still have a copy of that: computer rooms with the disk units wrapped in pvc sheeting because they were tearing out a wall directly behind them, a pile driver running right outside, rattlesnakes and skunks in the underfloor void (not the same computer room), basically everything that the environment and unthinking, incompetent and/or penny-pinching customers could throw at a hapless PDP or VAX and their peripherals

    [0] for government values of 'temporary', i.e. as long as it doesn't collapse by itself.

    1. Shadow Systems

      At StoneShop, re: temporary buildings.

      My Paternal GrandFather used to work for AeroJet GenCorp back when they made rocket engines & tested them on a weekly basis.

      He often did a lot of his "homework" in a tiny garage just off the house where he lived. Until one day the roof started leaking like a seive then gave way entirely under the weight of the standing water.

      His boss said they would pay for a "temporary structure" if he agreed to keep his "homework" under lock & key. The stuff he could take home was not Classified, but related to stuff that WAS, so they wanted that extra bit of caution.

      GrandFather said sure, "free building!", and was happy when the contractors arrived the next day.

      And put up an American Football Field (300+ Feet long, ~200 Feet wide, & over 20 Feet tall at the apex of the roof) sized quanset hut (looks like a metal cylinder half buried in the ground), covered it in ten meters of dirt, gave it power & water & a delightful aircon unit (the Grandparents didn't have it in the house, but GrandFather had it out in "the shed"), and left him with a "temporary" building in the early 1950's.

      My GrandParents have long since passed away, the property sold to cover bills, and when the bank showed up to look at what they had, the surveyor had an absolute *shit fit* over how the hell they were going to get rid of that "temporary" building.

      Seventy years of accumulated rain, wind, & snow deposited sediment helped to further bury the thing, cement the dirt in place, & leave it as closed to being a "hardened military bomb shelter" as a civilian was ever likely to have.

      They had to take it out with controlled demolitions.

      I wished I'd had the money to buy the place before they killed it, I would have used that "temporary" structure until the heat death of the universe!

      =-D

      *Hands you a pint, taps rims in toast, & drinks to those times when a temporary buildings being better built than the permanent ones*

      1. A.P. Veening Silver badge

        Re: At StoneShop, re: temporary buildings.

        There is nothing as permanent as a temporary solution, except for a temporary tax hike.

      2. zbcontent

        Re: At StoneShop, re: temporary buildings.

        Nice story! Over on this side of the moat, leftover WW2 Quonset huts are still going strong all over the country, and new ones are manufactured today, so if you want one, it's possible to get one.

    2. Alan Brown Silver badge

      Re: Not encountered a situation as extreme as this

      "[0] for government values of 'temporary', i.e. as long as it doesn't collapse by itself."

      One temporrary building I worked in was erected in 1939 and moved out of in 1986 - not because it was falling apart (it was) but because the concrete water tower it was tucked under started spalling chunks of cement - and then only when a 20kg lump went through the roof and partially demolished a rack before exiting through the floor

  7. Giles C Silver badge

    I remember being involved with a move of a storage array (complete 42U rack) from one office building to another.

    They turned up with a 7.5 tonne lorry and no tail lift. which I think had been arranged by the vendor.

    Somehow they got it in the lorry, and then unloaded it off a pair of ramps down into the reception of the building. Why nobody got a vehicle with a tail lift or maybe even hired a forklift I don’t know, all I remember is standing in reception hoping the thing didn’t accelerate as it came down the ramps as all that could have stopped it was a large set of glass doors...

    1. Tony Gathercole ...

      And contrawise

      Recall the time when working for the previous identity of a now well known Covid Vaccine Manufacturer, one of the systems - IIRC one recording adverse drug reactions - needed a significant (for then mid-1990s and just before SANs became generally available) storage upgrade. Kit was ordered by the application team and consisted of a significantly large (48U? and very wide) cabinet stuffed full of StorageWorks controller shelves, disk shelves and circa 150 4GB (could have been 9 or 36 its that long ago) disks - so you can imagine the weight of it. Muggins here was assigned to take delivery and arrange installation and commissioning.

      Anyway, come the day I'm waiting for the delivery all afternoon and eventually the truck turns up sometime after about 6 or 7 o'clock in the evening. Met the crew and directed them to the back entrance of the relevant computer suite and pointed out the doors through which the unit will need to be pushed. So, the crew parked up the truck and started to remove the packaging so that the unit can be rolled on its castors. They then pushed the unit onto the tail lift, fired up the engine and started to manoeuvrer the whole ensemble up the access road (backwards) towards the doors. Unanticipated consequence by the crew is that the storage unit is swaying side-to-side through frightening angles.

      Watching from a not-very-safe distance I'm by now having visions of the whole lot toppling over, falling off the tailgate and ending up with the mother-and-father of all disk crashes. So I'm shouting at the delivery crew to stop but to no effect. I can't now recall whether somebody jumped up and tried to stabalise the cabinet or what but somehow it didn't topple off and delivery was eventually completed, and the truck and it's crew departed - after me telling them their fortunes if the kit is damaged.

      Naturally, we kicked up a bit of a stink with the distributors over this incident and if I recall correctly it turns out we'd applied so much pressure for a quick delivery that they had had to use a non-specialist transport company and hadn't briefed them appropriately.

      We made sure that we ran a very full set of diagnostics and bedding in on all the kit before final commissioning!

      1. Stoneshop
        Facepalm

        Re: And contrawise

        Watching from a not-very-safe distance I'm by now having visions of the whole lot toppling over, falling off the tailgate and ending up with the mother-and-father of all disk crashes.

        Department of Agriculture was getting a new, integrated info and reporting system built, so that farmers would have one entry point for all the reporting they had to do: land taxes, subsidies, fertilizer and pest control use (because of environmental load trough runoff), harvest results, the whole lot. With bells and whistles on. Incorporating a nice GIS, too. So they need storage, and a beefy system to drive it all.

        An Alphaserver ES40 and a Storageworks box full of disks had been ordered, but as those tend to not deal very well with being dropped off a tail lift a new set was on order at the time I came to join the DBA to take care of the system management side of the matter.

        The truck driver had unstrapped the two racks, rolled them on to the tail lift, then found the truck wasn't backed up correctly against the loading dock. And even though it has been several centuries since Isaac posited that objects at rest stay at rest unless acted upon by an external force, this had clearly slipped the driver's mind. There were no forces acting on the racks as the truck mover forward, away from the loading dock until gravity, another entity studied by Mr.N, got to play with them.

        Result: two rhomboid-shaped racks going back to the vendor

        1. J. Cook Silver badge

          Re: And contrawise

          One does have to wonder just how many millions of dollars worth of compute and storage have been sacrificed over the decades to the Dark Gods of Shipping; even if you plaster the damn things with shock-watch stickers, 'tip n tell' stickers, giant "FRAGILE" stickers, invariably some chucklehead drops it off the back of the truck, tips it over, and/or impales it with a forklift. (or in one memorable case, ALL THREE- thankfully, it was a router and didn't have any rotating storage on it, but still.. a 2 million dollar core router being treated like it was a chew toy for Clifford the Big Red Dog by the shipping company did not make the boss a happy guy.

        2. ICPurvis47
          Facepalm

          Re: And contrawise

          Not IT related, but when I was working as a development engineer for a very large electrical manufacturer in the Midlands, we had a rush job to refurbish two cabinets of circuit breakers for BR Southern Region. The final assembly was being done in a bay we called "The Elephant House" because of its high lift capability, by our two Elephant Trainers, Sid and Sam. Because of a delay in procuring some vital parts, the build was not completed until late on Friday afternoon, and Dispatch and Transport were waiting impatiently outside with the Commer TS3 (sounded lovely, ever heard one on full song?) low loader. Eventually, Sid and Sam completed buckling up the cabinets, which were about four feet square and eight feet high, and lifted them with the overhead crane onto the low loader. Without waiting for any strapping down, the driver (who was anxious to get home) drove off down the yard to D&T. Everything was fine until he started to back it into their loading bay, which involved negotiating a slight ramp up from ground level to the building floor level. As he was approaching at a 45° angle to the threshold, one side of the semitrailer rose and tipped both cabinets off the other side, where they crashed to the ground, destroying all of the circuit breakers inside and distorting the cabinet frames. They were eventually lifted back onto the low loader and returned to the Elephant House, but, needless to say, they weren't delivered that week (or the next, either).

          1. Medieval Research Council

            Re: And contrawise

            Commer TS3 -- seems to be a sort of mini Deltic. Deltics sound fabulous and I often say they are the only diesel I know that sounds good. Any pointers to recordings of the TS3?

            1. Stoneshop

              Re: And contrawise

              Commer TS3 -- seems to be a sort of mini Deltic

              Hmm, more like a flat six but with opposed pistons, so three cylinders And instead of the conventional twin crankshafts outboard of the cylinders this one has the conrods driving some sort of rocker, then a second conrod from that to the central crankshaft underneath. Wooler tried a not-dissimilar engine configuration in a post-WW2 motorcycle. That one was two flat twins on top of each other, the lower pair driving the crankshaft directly, the upper pair attached via a rather demented rocker arrangement.

              Flat-(n*2) engines have a nicely low profile allowing underfloor mounting in buses and trucks; Büssing pioneered such engine configurations in Germany.

              Even a small Deltic, with its three crankshafts, would be a bugger to fit in a truck except maybe flat with the crankshafts vertical, but then you need an extra angle drive again.

            2. CountCadaver Silver badge

              Re: And contrawise

              https://youtu.be/GT4dprSxBiw

              Little bit of history there

              https://www.youtube.com/watch?v=p8k35_153Vw

              Also known as the Commer "Knocker" down under it seems.

    2. Alan Brown Silver badge

      "I remember is standing in reception hoping the thing didn’t accelerate as it came down the ramps "

      I find it doesn't matter if you DO give explicit instructions for delivery, they get ignored anyway.

      My staff are ordered to stand back and NOT assist deliverymen (invariably men) who are acting in unsafe manners. I don't want the H&S investigations that go with someone getting hurt and if they break something during delivery, it's all on video anyway.

  8. Anonymous Coward
    Anonymous Coward

    I remember getting an emergency call from a customer, their dial up comms wasn't working and they couldn't get their remote data uploads done to all their remote sites, not quite sure why but I took a very long full modem cable in the car before I set off. On arrival I investigated and indeed the modem software was failing to transmit anything. I must have asked if anything had changed and the response was 'we moved the computer room around'. Wandered in and sure enough every bit of (DEC) kit was in a different place, floor puller deployed I looked under the floor to realise they clearly hadn't disconnected any cables during the move and so the 3 phase power cables were now neatly wrapped round the modem cable multiple times. A quick trip to the car resolved the issue temporarily whilst they did the job properly. Probably very lucky that was the only issue that day.

  9. Scene it all

    I was working for DEC when the VAX first came out. Most of the system software development took place in Building #3 of the old mill in Maynard, Massachusetts. This building was there during the American Civil War and they made blankets for the Union Army. Everything except the outer walls was made out of wood, including the massive beams and all the flooring. The floor sagged in between the beams from the weight of the old wool-processing equipment, now long since removed. There were Maximum Floor Loading signs posted in various places. I remember quite a few said "25 pounds per square foot". I guess we were not supposed to stand with our feet right together. If you watch the movie "Pajama Game" you can see a building that looked a lot like ours, except inside we did not have flat studio floors, but wavy wooden ones. An advantage of a wooden building is that it is easy to run new conduits and there were actually factory "carpenters" who did this sort of work. Need a new staircase? Sure no problem.

    Anyway, the computer lab was on the 3rd floor and in the early days it was the only air-conditioned space in the building. The prototype and first-generation VAX 780 processors were very heavy so the building engineer did some calculations and directed that these computers had to be placed directly over the floor beams. This resulted in a comfortable amount of space between the racks, but no raised flooring so the entire room was kept quite cold. The very first 780 prototype had hand-built wire-wrapped connections. The next phase were manufacturing prototypes with soldered motherboards. These motherboards were huge - at least 2x2 feet each - and each machine held two of them. This is the hardware that VAX VMS 1.0 was developed on.

    So there was a trade show and it was decided to show off one of these. One was carefully disconnected, wheeled over to the cargo elevator (the only kind of elevator in those buildings actually) and taken down to a truck for the trip to the trade show. Everything was fine so far. But on the trip back from the tradeshow (as I recall it - I was not actualy present when this happened) either when loading the machine into the truck or taking it oiut, the whole thing was dropped. Not real far, but enough that the connections between the parts of the motherboards were cracked. All of them. II think it was a write-off.

    So we were down one development machine until the next one came out of manufacturing.

    1. Version 1.0 Silver badge
      Happy

      Ah, the old days! I remember having to fly up to Boston to "fix" a problem with a customers PDP11/34, it was easy, just pulled the main rack out and dived in with a wire-wrap tool and six inches of wire to reprogram the interrupts.

    2. Stoneshop

      The backplanes?

      These motherboards were huge - at least 2x2 feet each - and each machine held two of them. This is the hardware that VAX VMS 1.0 was developed on.

      At first I was thinking you meant the CPU boards but those weren't as large (2ft high and 1ft deep, roughly), and there were 20 of them, five more if you had the floating point installed as well. Then the memory backplane, nearly as big as the CPU backplane, plus the Unibus and Massbus interfaces, but those were narrower with 5 modules each IIRC.

      The backplanes still had a lot of wirewrap connections next to the board traces. There was a batch of flaky ones that had come out of a DEC plant in Canada; due to the machine's popularity there was a lot of pressure on production to crank out as much systems as they could, and some of the wirewraps were prone to being tight around posts they had to be routed around. Due to vibration from the fans over time the posts would cut into, and through, the insulation causing unwanted shorts. Once the root problem was identified production pressure was stepped back, and reliability went back to normal.

      Also, at one time there was an ECO that required a CPU backplane swap, the backplane wirewraps modified at one of the repair facilities, then swapped back with the associated CPU boards replaced as well. So the customer would have had to deal with two rather lengthy downtimes until one of my FS colleagues demonstrated that he could perform the backplane rewiring in situ and faultless. I think he had to show this to the engineering department in Maynard on three of their machines to receive their blessing on doing it that way. Now the customer had to plan just one, albeit longer, downtime.

      (DEC #201462)

      1. Scene it all

        Re: The backplanes?

        I think it must have been the backplanes I was thinking of that got damaged. It was over 40 years ago so some of the details are a bit hazy.

    3. StargateSg7

      So that means you saw Dave Cutler PERSONALLY when he was doing his VAX VMS development work? That must have been a treat! Did you get to see the start of the later 80's era PRISM / Mica Projects by any chance? I heard they had working prototypes of that super-worktstation CPU and OS in Maynard by 1987! That would have been a REAL TREAT to see!

      Had DEC gone ahead with Prism / Mica / Emerald, they would have OUTCLASSED IBM and even Microsoft by the mid-90's and DEC would STILL be alive today! They had the CPU designs and OSes equivalent to AMD Threadripper and Windows 10 in 1987 but Ken Olsen (head of DEC) was too stuck in his Minicomputer ways!

      We could have had 64-bit DEC Threadrippers and Mainframe-class Windowed OSes in 1995, more than 25 years ahead of Today! IF ONLY someone higher-up in the chain of command had listened to Dave Cutler and let him have his way!

      V

      V

      1. Scene it all

        I was in a few meetings with Dave Cutler, and also Dick Hustvedt. Hustvedt was a really smart guy who always wore a bow tie. I was working on compilers so did not interact with the kernel people a whole lot. Cutler did not write the whole thing himself - it was a team of people filling two floors of the building.

        I left that group long before PRISM came along. But also in my group was Carol Peters, who later led the West coast group that came up with the first MIPS workstations that were what eventually pushed the PRISM people out the door.

        More fun personally was a conversation with Richie Lary on a plane trip somewhere about compression algorithms. He was cool, and interested in everything.

        1. StargateSg7

          Wow! and DOUBLE WOW!

          You are ONE OF THE FEW who was in on the ground floor of hardware and software greatness.

          IF ONLY ONE decision had to be made by Ken Olsen ---- that SHOULD HAVE BEEN to let PRISM go ahead. I can only IMAGINE what we would have today in 2021 if we had a multi-core Gigahertz-capable 64-bit CPU WITH FULL VECTOR processing (i.e. SIMD and MIMD!) available by 1993 for the world's engineering and design communities!

          I'm SAD and MAD at the same time when SO MUCH POTENTIAL was there! We had wholeteams of DEC sales persons in Calgary espousing that "The Coolest" of things were coming out of Maynard by 1991! We liked/still like the VAX 9000 but I would have LOOOOOOOOVED to have 64-bit with vector CPUs in 1991/1992/1993

          Well anyways WE STILL use the COBOL compiler and EQUEL in 2020 so if that was you then GOOD ON YA for making something so damn bulletproof!

          v

    4. John Geek
      Pint

      my wife's first job out of college with her BS in Technical Journalism was at DEC in the old mills, working on internal and public documentation for VAX VMS 1.x, 2.0..

      1. Stoneshop

        Part of Ruth Goldenberger's team?

        I still have my "VMS Internals and Data Structures"[0] somewhere, though most likely in a box in the loft.

        [0] Or "VMS Infernals and Data Struggles" as it tended to be referred to.

  10. dvd

    We were installing computer kit in a bank in sub Saharan Africa and training staff. A data centre had already been built and the servers installed by the banks own staff.

    One of the banks head honchos decided that the plastering in the server room wasn't up to scratch and the walls weren't quite the right shade of beige. So the bank facilities proceeded to sand the walls of the server room flat, replaster them and repaint them with the servers running.

    We told them a few times that it wasn't a good idea, but the bank had screwed us about so badly that we were past caring so we didn't push it. As long as the kit lasted until our bit of the install and training was done we didn't care. I don't think that the servers lasted much longer.

    1. swm

      At MIT MULTICS project they had a "fire hose drum" for swapping. One day a plasterer working near the drum totally clogged up the filters and the drum shut down on over temp. "No problem," said the field engineer and removed the filters and went to get replacement filters. By the time he got back the drum was full of plaster dust and never worked again.

  11. Rufus McDufus

    This reminds me back in the 80s, I worked at a university nearby and we were asked to help out moving some gear from one of the grade-1 listed buildings near the Albert Hall. The equipment - I think probably some VAX 11/750s or 11/780s, though from memory a few things a bit larger (maybe a Cyber? I remember some impressive things), were in the basement of this very grand building. The staircase had to be encased in metal sheeting to protect it, and had a few corners. The gear had to go up to the first floor, with some help from motorised crawlers, and be taken through the window using a crane as the main door was too small and it was cheaper to take the large first floor window apart.

    It also reminds me of a hospital I visited in a tech support role. The poor onsite techie told me of the story building the server room (this was for a X-ray digitised record system, quite impressive for the mid-90s). They, presumably some not-very technical people, got all the server gear in the room, and then realised they'd forgotten to wire it for sufficient power . Wheeled it all out again, did the work. They then realised they'd forgotten networking. Repeat again. Finally, switched on and everything overheated. They hadn't put cooling in. Put cooling in. Cooling leaked and flooded the server room.

  12. Stoneshop
    Holmes

    A VAX8820

    does not need just a VT terminal, but a PRO350 console subsystem.

    1. Barking House

      Re: A VAX8820

      The cool feature was you could power on the VAX using the Pro350/380 via software only, no need to hit any power on buttons.

      1. Stoneshop

        Re: A VAX8820

        And you could query power status and a couple of environmental parameters, stuff that's now common. The later 88x0 machines used a MicroVAX II.

        The VAX9000 series essentially also had a MicroVAX as its console front end, with an RD54 that that one booted off. However, it was in an expansion cabinet to the right of the main CPU(s), so not immediately recognisable as such. The VAX11/780 had a PDP11/03 as the console system.

  13. a_yank_lurker

    Plugs (US vs UK/DIN)

    Many, many moons ago I worked for the US office of a German company. One of the picky points we had to pay attention to were the electrical specs for motors, electrical fittings (including plugs), etc. When I saw the wrong plugs were fitted I wonder what else in the power supply would be wrong.

    1. Anonymous Coward
      Anonymous Coward

      Re: Plugs (US vs UK/DIN)

      When St Catherines College, Oxford, was built in the 60s it was designed by Arne Jacobsen who designed/specified everything (even down to the cutlery used on high table - a design which turned out to require a subsequent commission of a small number of left-handed soup spoons!) and when the design was signed off the spec for "13amp square pin electrical sockets" everywhere passed without comment or checking ... however, when the building process reached the electrical fitout it was discovered that he meant a Danish plug with 3 square pins in line and not the UK square pin plug (there was an apochryphal runour that Jacobsen's brotyher owned the company that made these) and that this socket was integral to the architects vision for the buildings. Result was that students arriving were issued with a plug for these sockets to wire anything they need into .... and only after Jacobsen had died (as he had a habit of visiting the college to check the purity of his architectural vision was being maintained) did they rewire rooms to add in some UK spec sockets

      1. anothercynic Silver badge

        Re: Plugs (US vs UK/DIN)

        No wonder St Catz is a bit... odd. ;-)

        1. MJB7
          Coat

          Re: No wonder St Catz is a bit... odd. ;-)

          Probably because they can't spell Catharine properly.

          Yes, mine's the one with the St Catharine's College, Cambridge scarf in the pocket...

  14. Anonymous Coward
    Anonymous Coward

    "Ever stood by and watched a customer try to cram a square peg into a round hole?"

    Not quite ... but I remember way back discovering the reason that the Acorn Atom had only 5.75kB or program space when in the address map 6kB seemed more "logical". Turned out that while it was being developed someone (seem to think it was Chris Curry) was ordering cases for the keyboards to go with the System3 computers they were already selling but found to get a good price he'd need to order significantly more than he needed for that. Thinking laterally he had the idea of using the same case for the Atom which would provide the requried order size - so he checked the dimensions of the Atom's circuit board and the dimensions of the case ... and the circuit board size was smaller so everything looked good and the order was placed. However, at some point after this it was realized the case dimensions used were external size and there actually wasn't enough space for the PCB that had been designed. To get the Atom onto a PCB that would fit in the case would need an almost from scratch redesign ... apart from the pragmatic descision to remove a column of RAM chips so that the PCB width could be reduced at the cost of those 256 bytes of code space (yes, this was in the time when 256 bytes needed 4 chips!). Of course all the decode logic for these 256 bytes was still present and I recall articles in UK home computing magazines explaining how to regain these bytes bybuying the extra 4 chips, soldering them piggy-back on existing memory chips apart from the chip-select pin that needed to be wired to a the unused decoder output!

    1. Claptrap314 Silver badge

      Ahh. My only real brush with old-time hackery. The Amiga 1000 came with 256k on the motherboard and a 256k expansion available. More memory could be added externally, but this was quite expensive. As it happened, the main board memory did not fully decode the addresses, so in due time, there was an article in Amiga Magazine entitled, "Megabyte without Megabucks", detailing stacking additional memory on the motherboard, and including a short C program to type in to get the OS to recognize it. Lacking high-precision soldiering training, I had a buddy in my shop do that part. I then typed in the C program, complied it, ran it, and...nothing. The extra memory was not found.

      I don't recall why I decided that it must be the program that was the problem, but I had ANOTHER buddy with the kernel manual, which I borrowed. "What are all these asterisks?" "Those are pointers. The kernel is written in 'C'." So, off to buy a book about C...

      It turned out that the kernel had been updated, and the memory region structure now had three pointers which needed to be set, not just two. And I had my extra memory.

    2. G.Y.

      8008

      The 8008 was cut to 16KB in order to fit into a 16-pin package

  15. Anonymous Coward
    Anonymous Coward

    If it's on the 1st floor, why are you dealing with stairs at all? Some kind of screwy hillside building?

    1. Barking House

      In the UK the naming conventions for building floors is Ground Floor, 1st Floor etc (So in UK the first floor is the 2nd floor if you are used to the US naming conventions). A small but significant difference in the use of language between the two countries.

      1. H in The Hague

        "(So in UK the first floor is the 2nd floor if you are used to the US naming conventions)."

        According to the Dictionary of Scottish Building (Glen L. Pride, Rutland Press, 1996) in the past the Scottish convention was also to refer to the ground floor as the first floor. Unfortunately it doesn't indicate the time period for this usage.

    2. Anonymous Coward
      Anonymous Coward

      We index our floors from zero. Just to confuse Americans.

      1. John Brown (no body) Silver badge

        Like wot all proper IT industry people do, except in the USA. It's to screw with their minds :-)

      2. a_yank_lurker

        The US & UK, 2 countries separated by a common language - Churchill, I believe.

        1. ttlanhil
          Trollface

          Churchill is a common language?

          1. The Oncoming Scorn Silver badge
            Gimp

            Ironic as his mother was American.

            Icon - Have to go with Fanboi as I had a hardback edition of his life story as told by Eagle as a young boy.

  16. rnturn

    Basements might be as bad.

    Back while working at a University, the main data center where all the Big Iron (IBM 43xx systems), a roomful of DASD enclosures, and what seemed like 1000 miles of cabling lived, was located in the basement of an old building that had once been a post office (complete with Ionic stone columns). Removing and installing new equipment was accomplished by via a ramp laid down on a long flight of stairs. I never saw it happen in person but I'm told it was a process that involved a large team of people from the Uni's physical plant department following a procedure not far removed from the way ancient Egyptians hauled the stone blocks for the pyramids. I felt bad for the folks that still relied on those systems -- and would be down while that Herculean installation effort was being performed -- when the most difficult part of setting up our LAVC was running the coax into the offices and labs.

  17. Kernel

    "Ever stood by and watched a customer try to cram a square peg into a round hole?"

    Not exactly IT related, but in the course of my career I've worked on a PABX (BPO 100 Type) where the front of the cabinet was in a different room (and could only be accessed by a short walk down a corridor) than the rear of the cabinet (great for fault finding), a BPO 300Type PABX which was not only accessed via a long trek through the service tunnels under a hospital (watch your head on the concrete beam just as you are straightening up after ducking under the steam pipe) that had about 200mm cut off the top of the racks because the room wasn't high enough for the equipment and a rural carrier system that had to have around 600mm cut out of the bottom of all the racks because the engineers specified buildings that were designed for a GEC system, rather than the somewhat taller Fujitsu kit we were supplied with. I also remember having to cart two NEC x-bar switching racks to a rural exchange building to store them there for a future expansion of that exchange - once we got them there it was realised the the two racks from a recovered NC400 type were slightly taller than the building designed to contain a NC460 type.

    Sales people and engineers - you've got laugh otherwise the depression starts to set in.

  18. Anonymous Coward
    Anonymous Coward

    Nowhere near IT related but I did hear of a garden design/install outfit being engaged for a private garden. IIRC the price was circa £100k (how some people do live). Because access was discovered to be a bit limited (I dont have any more info than that), said price tag included £6k for the services of a helicopter to ferry in half a dozen trees for planting.

  19. Stuart Castle Silver badge

    My old uni.

    They had a state of the art, actually quite posh as it (along with most of the building) had been refitted a couple of years before, gents toilet. Complete with one very wide and very well equipped disabled toilet, with handles all over the place. The trouble is, the toilet itself was only accessible by a staircase.

  20. Stuart Castle Silver badge

    When my company took over a building, the architects put some of the that staff in incredibly small offices. One that I used to work from time to time was barely big enough for one table and two members of staff. At least two of these offices also had no windows, and barely any ventilation. Not good considering the users were technicians and used a lot of equipment). Odd then that they set aside two large rooms at the end of the corridor that were well ventilated and had large windows (so a lot of light) for network patch panels.

    To make matters worse, they installed the patch panels in the centre of each office. So we couldn’t merely partition the offices and use them for people as well.

  21. NITS

    Some decades ago I worked for a company that provided transaction-processing systems for the convenience of punters wishing to invest in equine performance futures (a.k.a. on-track wagering systems, a.k.a. "the totalisator"). The equipment was housed in heavy-gauge welded-steel double-bay cabinets with casters underneath, able to be moved overnight if necessary from one venue to another; the company had its own fleet of air-ride semitrailer trucks for this purpose. Some customers' sites had loading docks, others required the equipment to be (un)loaded via a portable ramp to the parking lot, then rolled into the building.

    The prototype systems had used fixed-head disk storage, which was found to be less than reliable, especially when transported. Later systems were fitted with Ampex core memory boxes. But even those were found to fail in transport. So an experiment was planned: Fit a system with shock-indicating sensors, transport it on local roads and freeways for 50 miles or so, and bring it back to the shop to see which (if any) of the sensors had tripped. We installed 3 sensors per cabinet, one each calibrated to trip at 5G, 10G, and 15G acceleration.

    The experiment was a short one; all of the sensors tripped in the course of rolling the cabinets across the expansion joints in the shop floor, and over the dock plate into the trailer.

    Failure of core memory planes remained a problem until bulk semiconductor memory boxes became available a few years later.

  22. IGnatius T Foobar !

    "Right," said Fred.

    Please tell me that was a deliberate pun.

    1. TSM

      Re: "Right," said Fred.

      The line immediately before it links to the video on YouTube that it is referencing.

  23. Imhotep

    I built a shed once.

    Just one? That's cute. I built two!

    Arnold Jackson

  24. yetanotheraoc Silver badge

    Right Said Fred?

    Check.

  25. chrishansenhome
    FAIL

    Weighty FAIL

    I was the Testing Manager for a company that required very big iron as a testing platform, sourced from a very Big Blue company. The model number is immaterial as I can't recall what it was now.

    I set out my requirements, and got a proposal from the source. As my proposal traversed the various levels of the company I was working for, the supplier got antsy. The salesman called and invited me to a very important football match. As I'm not a fan (I know, right?) I declined politely.

    I had consulted the local BOFH while preparing the tender document, and he had given it the A-OK. However, at the very last minute, he had discovered how much this Very Big Iron actually weighed. So, in true BOFH style, he smugly informed me that he'd calculated that the machine room floor was inadequate to support the weight of the machine without reinforcing the floor at an unspecified but horrendous cost. This put the budget for the machine over my budgetary limits. We had to inform the Big Blue Company that we weren't going ahead. Luckily I could pass that task along to my boss.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like