Don't let that guy work without supervision
Greetings and felicitations, dear reader-folk, and welcome to the soft landing to the working week that we at The Reg call Who, Me? in which we share tales of readers like you who found themselves in unfortunate circumstances – often of their own making. This week's tale is a lamentable one, to be sure, as it is a story of …
I don't have a grey beard, but I remember having to go to Boots with a disposable camera or a roll of 35mm (actually my first film camera was a disc format!) and handing it in, waiting days to get it back. Then digital cameras happened and I didn't bother with film.
Until 4 years ago, where I bought a cheap russian 35mm camera and a roll of Ilford XP2 (black and white, but can be developed with the colour process). I've not used a digital camera since for serious photography. Don't get me wrong, I whip my iPhone out to take a photo if I have to, but if I am on holiday or just fancy a day of taking photos, I will use film.
Why? The development of it. It's alchemy mixed with meditation. It provides excitement at the same time. Load the film in to the container, mix the developer, swirl it at set intervals (or just leave it for 1 hour for interesting results), stop bath, rinse, leave them to dry. Then stick them in the scanner while taking a sneak peak of your shots looking at the negatives. Only reason I scan them is because I don't have enough room for a dark room, but if I did I would probably do that as well.
On paper it's an awful process. It's time consuming. It's expensive. It's not instant. But given we live a life of being in a rush and having instant gratification, film photography offers something different. Like I said, it's like meditation. Plus, given you have 36 chances to get a photo right it forces you to look at something and decide if it's worth it. Before I got in to film again I would happily snap away on my digital camera and I would get maybe 1 good photo and the rest would be garbage. And would I even look at that one good photo again? No, it'd stay on the device to languish with the other photos I took in the heat of the moment that never get looked at again.
I too never used stop bath on film, only on prints. For (BW) film I'd do developer + agitation, rinse, fix, rinse thoroughly, squeegee.
My father, who is an avid photography enthusiast, told me when I got started that you can't go wrong with Tri-X film - the worst thing that can happen is that the emulsion itself cracks, and if it does it'll crack absolutely evenly and you'll get a cool graphical effect from it. Well, I proved him wrong when developing my first roll in the school lab one night. I did the last rinse in WAY too warm water, so when I went to dry it off, large swaths of the emulsion came off the substrate and stuck to the squeegee. It wasn't very even, but the resulting pictures were pretty ghostly and kind of cool anyway. So I guess half a point to dad for that one.
I can still remember the smell of Agfa's Rodinal developer combining with the 1:63 dilution of Kodak's Indicator Stop Bath. Fondly.
I use a stop bath for film developing and processing. I heard on a YouTube presentation that the standard chemicals will process two 8x10 sheets of film, or their equivalent, which means that my two developing tanks (one for roll film and one for 4x5 sheet film) will actually process twice their capacity. So I load up the tank, pre-wash the film ( Ilford HP5+) develop, stop, fix and wash, while saving the chemicals for another round. The stop bath does help to preserve the efficacy of the fix solution. It works. Now, admittedly I'm unlikely ever to make images as wonderful as Evelyn Hofer (current exhibition at the Photographers' Gallery in London well worth a visit) but hey, it is good fun.
Evelyn Hofer: https://thephotographersgallery.org.uk/whats-on/evelyn-hofer
Her still life photographs are amazing (link too long, just search)
I'm using an Intrepid Cameras 4x5 Mk4: https://intrepidcamera.co.uk/collections/camera
(Yes, I am retired and have oodles of time, and a bit of spare money.)
Never heard of her, but she reminds me of Vivian Maer. I heard about her story (well first heard the Manic Street Preachers song about her, then deep dived) and I find her fascinating.
There's also Phyllis Nicklin who did a similar thing in Birmingham, as well as Vanley Burke who I've met and he's a wonderful man. There are so many people still documenting life around us and make great photography from it.
I once did a media training course (one of those get naïve-and-lost young
graduates people off the dole figures things (and it was genuinely useful, I hasten to add (and probably much more of a confidence boost and help to find my feet than my university had been, where we were very much treated just like fodder))), and one of things we got trained up in was photography.
Yes, long enough ago that there was no D in SLR, and we were also trained in how to develop the shots we had taken, in a darkroom. It was all rather good fun (and I took quite a few shots that I'm fairly proud of), but the whole darkroom kerfuffle is not something that I'd particularly want to have to do on an ongoing basis - kudos to those who do enjoy it, however.
The photography trainer was a really decent bloke and definitely taught us some useful skills about how to frame shots and make good use of focus and exposure, etc, but I later heard that he left a year or two later to become a crime scene photographer with the police (I think more because although the media centre job was enjoyable, the pay wasn't so great - I can't imagine there would have been so much creative freedom in his new job, however, and I'm sure some of it would have been rather grim!).
I had a course in middle school (6th grade?) that included taking black-and-white photos, developing the film ourselves, and making our own prints. Not sure I'd want to do it routinely now, but it definitely gave me an appreciation for the process. The teacher had us wind a scrap bit of film onto the developing reel, first in front of ourselves in a regular room with our eyes open, then behind our backs until we could get it right without being able to see it. Only then could we attempt it with "real" film in the darkroom.
Just checked Wikipedia now, and it seems that both Tri-X and T-MAX are actually still being manufactured by Kodak themselves (and sold by Kodak Alaris after the collapse of the Kodak empire). It’s been many years now since I shot on film, but I loved experimenting - and having a father who worked for Kodak in the 80s and 90s definitely helped feed the addiction…
The colours you could squeeze out of the Ektar 25, the definition of the T-MAX P3200 and - as you say - the Tri-X grain… Add a Cokin P057 Star 4 filter and I’m happy.
Nah, in this case they're both the same.
One person has a hobby they get a lot of enjoyment out of which also has the side effect of producing something potentially beautiful.
Other person doesn't understand this and feels the need to dig, for some reason.
Power to the photographer I say.
(Also, as a currently retired, take advantage of it, you're in that golden age where you'll actually get one.... I'm in my 40s and will be working until I drop...)
" You can see that a decent photo is good, but a lot of audio nonsense is just hocus pocus woo-woo."
Some of it yes. But most of it exists solely because people haven't had an opportunity to actually listen a good audio system and they have no idea.
Good ones are in tens of thousands of pounds (or euros), very few can afford to own one. A good (film) camera is *a lot* cheaper. But both have one thing in common: At high end price has very weak connection to picture/sound quality.
It's true. If you spend tens of thousands of pounds, you will (probably) get a good audio system.
But you don't need to spend that much!
A couple of thousand, concentrating on the turntable, cartridge and speakers can get you something that will exceed most people's needs (and their preparedness to arrange a room for optimal listening!)
If you spend very carefully, and cherry pick what you get, especially if you're prepared to buy second-hand, you can get something very listenable for under a thousand, and if push come to shove, I could probably put together a reasonable sounding complete system, new, to play LPs, for under £500, but there would be compromises (the speakers, especially, and you would end up with a turntable sporting a AT-3600L cartridge which is an absolute bargain).
My vintage kit, which currently includes a modded Pro-Ject turntable(1) with a classic Ortofon cartridge(1), Cambridge Audio CD player(2) and DAC(2), rebuilt NAD receiver(2), ancient JVC casette deck(1) (this is the last component of my original HiFi that I'm still using), and a range of speakers from Kef(2), Warfdale(2) and a niche brand called Keesonic(1) (I use different speakers for different types of music) could probably be bought in it's entirety second-hand for well under a grand. I've collected it over 40+ years, some new(1), some second-hand(2), replacing bits when they break or I find something better, and I think it sounds more than good enough (although the itch is always there), and it has certainly surprised visitors when they used to come into the house.
I actually have more than a complete system of components I've tried to see if I could improve the sound over the years. Must get around to advertising some of it to sell while there is still a market for them!
I don't bother with vinyl, it takes up space, has an *archaic* UI, wears and so on, so I mostly deal in slightly smushed bits. Yes, I know vinyl can sound good. It's just a hassle and my crappy ears (I'm 50, have tinnitus *and* 30 dB drop in treble in my right ear) can't hear the difference between the original and a version with strategically lost bits.
Those who buy expensive audio equipment are very sure that doing so has an effect, whether it does or not. The same applies to people who have very expensive or specific photography equipment or processes. Both of them are correct to some extent, as a moderately expensive camera or speaker system is better than the cheap versions available to anyone, but either can be taken to extremes where they're convinced that the product is better even though everyone else can't tell the difference. This doesn't matter much to me, as I'm not doing either so whatever equipment is used doesn't bother me, but it can sometimes help to consider whether some peculiarities in your desires might be approaching the unrealistic level that you see clearly when it's something that's not your hobby.
Hey, look, my OS and monitor can do 128-bit color? Still enjoying your 32-bit (which is really 24-bit)?
Oh, wait, actually, it would look exactly the same; once you're in the "millions of colors" range it's hard to see a difference. I imagine it would be the same if I had a monitor with twice the resolution of my current 1920x1080 - the limitation rapidly becomes the eyeball rather than the hardware.
But you can tell the difference between the types of guitar, LP, Strat, Tele, PRS, Ricky... My beloved RS sounds nothing like my Squier Strat, even on the same amp settings.
And you can tell the difference between players as well - Dave Gilmour and Steve Rothery sound different playing the same piece of music, for example.
Hey, my "hobby" is riding my electric bike to places I want to go.
It's a pain in the ass (literally - I'm buying a better seat) to plan out the route and all the charging stations, but I kind of enjoy that. It's like planning an airplane flight, and I sure as hell can't afford an airplane.
I can see the same sort of satisfaction from developing your own prints.
I get the impression that some people have been brainwashed by the low resolution of digital cameras and don't realise what photography can actually do with a large format. The information in a large negative would be hard to match in digital although Leica did,as I recall, try setting what was essentially a scanner mechanism on the focal plane of a large format camera. The time needed to set up a shot on a large format means that the result reflects a degree of thought denied to that of a point and click.
Low resolution? That's not what I'm hearing from the photography pros. I've heard a lot more about how a 50 megapixel sensor just isn't going to help without better lenses than the cheap stuff, new or used. Even sometimes that 12 megapixels is plenty for virtually all purposes, and that is better than 4K, which is only 8.3 megapixels.
You can get medium format or large format digital cameras. Most of us don't have fifty thousand dollars to drop on a camera like the Hasselblad H6D-400c, and few ever have. I don't know who is being brainwashed, but most people have gone from compact point and click cameras to cellphones and are happy about it. Some carry DSLRs like we might have carried SLRs in previous days. There have probably been more people with tools and skills to copy a scene in oil paints than large format camera, both then and now.
As a student in the 80s I worked in production engineering for a small electronics company. They did their visual aids by taking B&W photos of the assemblies then colouring in the relevant components in felt tip to highlight them. So if the VA was for RTVing a capacitor bank I'd take a before pic of the capacitor bank with someone holding the RTV gun in the right position, then take after pics of the finished assy. They had a darkroom on site so I learned to develop and print then I'd use felt-tip pens to colour the capacitors in blue, the tie-wraps in red and the RTV in yellow then write the instructions. The VAs looked really good and the colouring made the conmponents stand out really well.
I took a conscious step back a couple of years ago with digital photography. Like most people* I got into the habit of shooting lots of pics in the hope I'd get a good one rather than thinking about, and setting up, the shot carefully as I used to do when shooting expensive slide film. I've stayed with digital but tried to apply a film mindset if that makes sense.
The one thing I would go back to film photography for is B&W - for some reason digital monochrome, whether produced in-camera or post processed, just never has the same look.
*I met someone at the North Norfolk railway who shot absolutely everything on 4K video then spent days searching through for the few stills shots he wanted!
Like most people I got into the habit of shooting lots of pics in the hope I'd get a good one rather than thinking about, and setting up, the shot carefully as I used to do when shooting expensive slide film.
Aaron Johnson noted this phenomenon in his fantastic What The Duck comic strip (2006-2016). Forgive the links to the merch pages, but the comic archive search comes up with 0 hits.
I went nuts a few years ago and bought (very cheaply) some of the film cameras I had lusted after as a kid. I still use them from time to time.
I discovered that I work differently with film. You become conscious of the "cost" of the medium; not so much in dollars (or pounds), but in the time and effort required to see what you have exposed.
I'm not sure if the instand results with digital help you to "see" what your film work will look like, or if the additional deliberation that comes with film photography makes you a better digital photographer. The two do seem to complement each other, though. Of course, today, I don't think anyone still makes prints with an enlarger...I scan my negatives and print from jpg. I do still have the equipment to process 35mm and 120 film, which I do occasionally. Just to keep my hand in :-)
Antron Argaiv: Of course, today, I don't think anyone still makes prints with an enlarger
Some people still make enlargers, and sell them:
I learned to take photos on film. Because of that (and that you had ~30 photos in total which cost you money for both for the film, and getting them developed) you learned to make sure that you had a decent shot, that you'd got the settings reasonably right and weren't making any stupid mistakes like taking photos on bad angles for lighting (eg, into the sun or similar)
The result is that with a digital camera which sorts out a lot of the settings for me, I take excellent photos because I sort out angles etc instinctively and "know" if i've got a shot before touching the button.
I have a few hobbies which result in me bumping into photographers. Out of those, there are only two who really outclass me in every way in equipment I'm frankly envious of, and knowledge. Having a chat with the better one (cadging some tips in the process) he suggested that having lived through film photography was a training experience worth considerably more than the college class in photography that many of his competitors had. We were saying this watching an aloof know it all "professional wedding photographer" who was wandering around after us trying to duplicate our shots (which were mostly close up portrait photos) at an event. (probably trying to implement "if you can't beat them, copy them")
The thing is, we were both using 15-55 mm lenses which are suited to close up work, but can't do distance photography worth a damn (hence us wandering around taking portrait photos.) This chap copying us was using a 200mm+ telephoto lens which can't do portrait photos worth a damn, but can take very nice photos (looking like a closeup) from a few hundred meters away. He needed an entirely different set of angles and shots for his equipment, but didn't have a clue how to use his equipment, and of course with no rather harsh financial penalty for taking bad photos (which used to be inherent to film) there is nothing to stop people from just keep snapping away until more or less by random chance they get a half decent photo.
professional wedding photographer
We were at a hotel last year (the Dunblane Hydro) where a wedding was taking place. They had three wedding photographers plus a videographer wandering around - I can't comment on the quality of the results but that seemed overkill to me! And yes, long lenses seemed to be much in evidence in an environment more suited to something sensible. perhaps it's what they're taught in Wedding Photography 101?
A mate used to delight in visiting those photo shops that had the developer/printer in the window with a roll of film from a debauched night, out then waiting in a coffee shop opposite to seen the expressions of the passers-by as his prints came out of the printer
For some strange reason he never actually paid or collected any of them...
I didn't grow up in the film era of photography, but I do remember the times when CF or MMC cards had limited storage (or using the camera flash drained your battery).
Not having to worry about battery life or storage capacity is a good thing, but in my opinion the scarcity of photos you're able to take does increase their value. You make sure the shot is perfect, and only take pictures of things that actually matter.
These days I spend more time sifting through trash photos than enjoying memories.
I take loads of photos (many of them my little furry friend), and then spend ages looking for just the right one some time later.
But what I've found is that all too often, the photos are "flat" compared to the reality. Things that looked impressive create somewhat meh pictures.
But more than that, put the bloody camera down or those photos will be the only memory you have as there is the danger of spending so much preoccupation with the camera that the real event is simply missed. Experience first, photograph after, but that latter part is optional...
At the time of the eclipse we went to Cornwall. One child was a toddler, the other waiting to be born.
We all waited on the edge of a cliff for The Moment. The sun got darker ( being by the sea was a mistake by the way, but that's another matter)
At which point Mrs 6 shouts "I left the camcorder under the pram". By the time I found where she'd hidden the bloody thing I'd missed most of the eclipse.
I guess I'm one of the last who grew up in the film era.
My dad owns an Olympus Pen EE-2 with a cheapo Soltron Flash. Got it for dirt cheap from a friend who was leaving the hobby. It served him extremely well.
We got onto digital photography very late. I got my first digital camera in 2003-ish. It was a cheap Chinese brand camera and the pictures it took were grainy and crap, but it made me happy.
Nowadays the whole family uses the camera on our phones now. No one carries around a camera anymore.
I'm really tempted to break out the museum piece and see the Olympus in action again, although I think camera shops most likely won't know how to make heads or tails of the picture it takes...
It's a Unix system....
HPUX from the shutdown command but I would think the root shell prompt would be '#' rather than '>' unless it was to indicate it was the backup server having taken over the production role :)
I always liked the BSD 'fake' shutdown option (-k) with 5 mins grace (to get the users off or avoid Han's mistake) followed with a real shutdown.
I always liked the BSD 'fake' shutdown option (-k) with 5 mins grace (to get the users off or avoid Han's mistake) followed with a real shutdown.
I set-up our UPS low-battery shutdown alerts to report to everyone the system will go down in 3 minutes, though it's actually scheduled for 2 minutes.
I hate procrastinators.
"To be fair, most of us don't have a simulation country tucked into the desk drawer ..."
Doesn't quite fit into your desk drawer, but this should do nicely:
An analogue fluidic simulator.
"the only way to test the output of a change is to actually deploy it in production."
One program in the system I looked after had been written by a programmer who, thankfully, had left long ago. It always annoyed me because of her odd programming style. It largely consisted of much the same code repeated multiple times. It was ripe for refactoring as we say now but maybe not at the time.
Late one afternoon I decided to tackle it.
One block of code was repeated several times. Copy that into a function and replace all the repetitions with a function call. Very straightforward. This must have shrunk the LoC to about half.
The remainder of the repetition consisted of two similar but not quite identical blocks of code each repeated several times. Not quite so straightforward. Copy one version into a new function adding a switch parameter then add the different sections of the version using the switch parameter to decide which t call.
Replace the repetitions with function calls taking care to use the correct value of the switch.
We now have a much simpler program, a fraction of its original size. It ought to be easier to understand what it's supposed to do which was one of the objectives of the change. But the remainder is still a bit of a tangled mess. Sorting through it to work out just what data would be need to test all the alternative paths would still take ages.
Well it was really all very much a mechanical replacement - the same code is being run, just from one of the two new functions instead of inline. Of course it must still work exactly the same as before.
It was now early evening. Why not put it live?
So I put it live.
Of course it worked exactly the same as before - what did you expect?
Decades ago, late 80s maybe, as an amateur programmer I read lots of articles explaining stuff like how to write functions in the code.
None of them bothered to explain waht ehy were for or, in effect, why we should do this.
It was a couple of years later, when I was working for a living and had pretty much stopped writing software that I was polishing up some rather poor BBC Micro educational software to make it more usable ( OK probably breaching some sort of copyright) that I saw some function calls in action ( the coding wasn't bad, just the programme design was appalling, if my memeory serves me correctly). A bit of a lightbulb moment.
My ex-college room-mate was talking to me a few years after we graduated about wanting something to help his (secondary school) students prepare data from experiments, using the BBC Micros that the school used.
I took this as a challenge, and wrote him a program that could take multiple data sets and plot them against auto-scaled axes, with the ability to load, save, print, edit and even merge 2 different data sets together, along with the ability to apply a function to the data before plotting it. Not as extensive as something like Gnuplot became or SPSS, but something suited for the classroom. (and this was something like 1983).
I was working in education (further education in my case), and had been writing demonstration code (mainly in ISO standard Pascal) that HNC/D computing students could use as examples of good programming style, so I decided to make this BEEB program a Rolls-Royce of coding, on the assumption that the kids would take it apart to see how it worked, and I wanted them to have as good an example as possible. It used every good feature I could think of that BBC Basic provided, and even though the memory was tight, managed to leave enough comments in so people could work out what it was doing.
I got some very complementary comments back from staff at my mate's school, with their IT technician saying that he had never seen such a well-written program. Strangely, a program with very similar capabilities to mine appeared in educational software catalogues soon after, but as I had explicitly written in the comments that I was making it freely available, I don't think I could complain (not that I proved it was my code).
I keep meaning to find it again to see whether it stands up against my memory, but it's buried somewhere in the hundreds of ageing 5.25" disks that I accumulated over the years. I was going to go through them to write them to SD card (I bought an SD card filesystem for my BEEB and refurbished it and another one just for this purpose) before they became unreadable, but it seems such a huge task.
Of course it worked exactly the same as before - what did you expect?
Haha! Nice non-twist in the tail Doctor Syntax!
Sounds like you produced what I call "ravioli code". It's more palatable than spaghetti code as the functions form nice little enveloped pouches, but they're still floating around in some disorderly sauce code!
[Edit: I've been using that phrase for decades. Doing a search reveals I'm not alone - I really didn't know it was out in the wild!]
I've never heard it , but I've long been of the opinion that spaghetti is the stupidest possible shape for pasta , or indeed any food , so I can get behind it!
... and to make it worse when you go eat at an italian , they confiscate the knife! lest you try to chop the spaghetti into manageable pieces :(
I'm glad you liked the twist.
Sounds like you produced what I call "ravioli code".
If your analogy is correct there would be just the same amount of code there, just packaged differently. The main object of the refactoring was to reduce the LoC and hence the amount of memory consumed at run-time.
If I was the original programmer, I might use that style to deal with, say, ten different cases that I thought were going to be mostly just the same, but I wasn't sure, or I thought there could be a requirement change, so I put the same unit ten times with variations in the copies that required it.
Or if I was afraid to use functions, or if I wanted the program laid out neatly on one page in the order that it happens.
Or if a manager wanted the program to be on one page and in order, without subroutines.
But also, gosh, I've written some clever stuff, that I don't understand myself at maintenance time.
When he arrived he confirmed with the manager on duty that staff were aware of what he was doing. A few staff were using the Backup/Test server, but he should be OK to shut it down in about 15 minutes.
Ok. He told them about the plan, they said this was good with them, even telling him that some people were using the backup/test server. Sure, he could have checked the number of active connections to the machine (confirm stuff - like when working on electrical installations: you check for yourself the fuse is out and secured against being turned on again), but the production machine could have keeled over "just then", basically when he shut down the test system. It would have been better to have both a backup and a test system, but who has the ressources for that... (I hope that manglement learned this lesson at least).
I did wonder why no one at the company had noticed that they were running from the back-up system, and told him. Why didn't they realise this? A properly configured system should have sent some sort of alert when the live server failed and the back-up server took over, I would have thought.
Having worked on power-electrics your adage rings so true.
"Is it switched off?"
"Is it isolated?"
"Is it earthed?"
"Can it be back-fed?"
No. No. No. Yes.
I was so paranoid about this, if I was isolating something for someone else, I demonstrated it was safe by putting my hands on the busbars. A tip gained from a North Sea Oil worker.
I once got a serious belt from the HT side of a 5kV - 240V step down transformer that I had been assured could not be back fed - it could and was. In my agony I managed to scream hoarsely "Switch the ***** thing off" and one of my co workers dived across the board and killed it before it killed me. This led to an in-depth investigation as to why, but I missed that because I was off sick for a week to get rid of a nasty case of the shakes. When I returned to work, I always shorted anything out before I touched it, sometimes with spectacular results.
Just literally had this, get into work.
See the test cell being worked on,
Walk up, halfway there "Ahhh just the chap, you are good to go on Test Cell updates"
Get back to desk, check a couple of e-mails (Outgoing PHB (Terminated) boss) & start my script running!
At that halfway point chap who gave me the go ahead goes "Ohhh can you not do the vibration server yet, someones still working on it"
"Sorry - Too late bud" literally as my powershell script reports that its just completed the restart process to kick off the ghost image & starts on the next PC.
"Ok. He told them about the plan, they said this was good with them, even telling him that some people were using the backup/test server."*
Before the other shoe fell, I figured the fail over would have happened in between issuing the shutdown command and making his way to the donuts. It would have been nice to be brought up to speed*
nonsensical that they didn't wait till after hours.
Hey boss shouldn't we wait until the end of work day to start screwing around with the servers?
Boss says, but then how would I get my adrenaline fixed from risky behavior? No we're going to do this the exciting way.
One of the most pathetic letter I received was from a film developement business. They had used to send ultra-glossy magazines, boasting the quality of their colors and their crisp detail. This one was a single black and white sheet of paper, acknowledging that they were mostly dying since everything was going digital, but that maybe we still had old films which we'd like more copies of, or perhaps even digital pictures to print, and they were fully ready to do the job. Like receiving a letter from your grandmother in the hospital saying that maybe you can visit her, whenever you have time, while there's still time...?
I found about 10 old rolls of undeveloped film in boxes while having a clearout and decided to have them developed to negatives as I have a negative scanner packed away in another box.
They came back with a warning that because the films were old and may have been stored in less than ideal conditions they could be degraded. Some were, but I could make out what they were. Others were fine, and the latest ones that were ok were from my daughter's 4th birthday party - she's now 31!
I once supported a certain database software, and the customer was a certain Hull-based telecoms company. I'd been given a number to dial in to access a test server to do some work on it. I dialled in and immediately saw reboot messages displayed. Hmm, weird I thought. This may have happened a second time on a subsequent login. Anyhow, later on I saw it reported that this telecoms company suffered some outages...
Colleague did the same thing only without the prior check - just happened to patch the live server first. Cue howls of outrage telling him to always do updates on stand-by first, test and failover and only then do live.
Thus assured in clear messages which left no room for doubt that he was patching live in error, he had learned his lesson, knew better and deliberately patched stand-by this time...
...which by now had taken on the live workload, causing yet another chorus of howls.
Oh God. Back in 2002 (at my current employer) we had a bespoke-built Intranet system on a physical Dell server and a copy on a PC (same OS, in theory identical) as the test server.
Microsoft patches. Worked fine on Test. Rolled out to Live. Irretrievably broke the Live server which needed complete rebuild plus a full rebuild of the bespoke software at considerable additional cost.
This will now identify me to approximately three people in the world.
Reminds me about the story I once heard about a system that failed.
Supplier got ripped a new one as the customer had purchased a fully redundant with failover/bells/whistles the lot and nothing should ever go down. Turns out the primary had failed a while back and no one noticed until the secondary failed.
About 35 years ago I looked after a file-server* that was installed in an office on a mezzanine. Unfortunately, the mezzanine also provided enough space for the component stores used by our electronics company. In general we co-existed reasonably well, apart from one young storemen who decided that he should run up and down the stairs as quickly as possible. (No 'elf and safety back then).
Being a bit of a skin-flint operation, the mezzanine was done on the cheap and developed a bit of a sway as people moved around. After about 6 months the disc drives started getting louder as the bearings were being knackered by the constant motion caused by the lumbering storeman. After many hours of negotiation, and prophesying an impending complete system failure, I eventually managed to convince the MD into getting the fabricators back to weld in a few cross-braces on the mezzanine supports, and score enough cash to buy new drives with a higher capacity.
Eventually we were able to create a dedicated room on the ground floor after we'd made enough profit to buy the factory next door.
* A Tandon 286 with two (count 'em) 20MB MFM drives providing storage for 6 or so terminals (IBM XT Clones) running our very basic ERP system (Multisoft, anyone?) Everything strung together with 75 ohm co-ax, and DOS 3.31 as the network operating system. Ah, simpler times.
Oooh. Naughty naughty. Embedded terminal control sequences.
I suppose most people use something that looks like an ANSI 3.64 (ISO 6429) terminal now, but I always get jumpy when I see such things as not all terminal emulators implement the same colours or control sequence supersets of the standard. In addition, my experience with the new MS Windows Terminal indicates that this is missing several capabilities that appear on an xterm (which is how it identifies itself), so things like vi are unusable. Hopefully fixed in later versions.
Just started a new job about a month ago at a company migrating to SAP. A week or two back I came in to hear people talking about how some new data center admin from SAP decided they would be helpful by getting rid of a few things the company wanted removed. So they ran some kind of script that removed that data, and a whole lot more, and naturally the script was set to make sure there were no backups made or anything like that.
It’s why I always make sure to do test runs when working with tcodes like mm17 and treat them with the respect that is deserved of a tcodes that can make mass updates to materials. I make a very conscious effort not to get complacent with any kind of mass update tool.
When I first started work, we had a 360 DOS/VS system. Our development team had 2 disks. One live, and one build. After the build was successful - they swapped disks.
My job was to compile the product onto the build system. My second day in the job, had people coming round saying "its stopped working". Whoops I had built into the live system. My team leader came and discussed it with me, and agreed I had followed the written instructions - but there was one line missing. "Mount the build disk". He edited the instructions.
Next day the same problem happened - I overwrite the live system. My team leader came round... and agreed that yesterday's build disk was today's live disk, so the documentation should be clearer. He moved me to a different area where I could do less damage.
This reminds me of an Instagram clip (yes, I know, shut up) where a father follows the instructions given to him by his child to make a PB&J sandwich (for the non-Americans, peanut butter and jam).
Here's the Youtube clip of it. It's brilliant.
Probably yes - Han's problem is (as reported) that he got the backup server to turn off, just after it became the live server.
But conceivably, the live server hands over to the backup server only if the backup server is there - say, when a problem is not yet critical. In that case, it would have been better to remove the backup server sooner.
People swap machine-guts without updating the physical computer-case labels.
People swap machine-guts without updating the disc labels (physical and/or electronic).
People swap machine-guts without updating the hostname.
People swap machine-guts without updating BGINFO.
Co-worker during Y2K testing: "What's wrong with this monitor? The screen background is red instead of green.
Me: "There is nothing wrong with that monitor. PROD is red, TEST is green, and DEV is blue."
Co-worker: "But the screen is red, yet it reads TEST."
Me: "Did you do anything potentially data-destructive at that keyboard?"
Co-worker: "Uhh ... maybe."
(Icon for the work required to sort out the cock-ups caused by careless/lazy people taking shortcuts ... and not telling anyone)
I once pulled the backup processor in a system in a test lab, that was absolutely not connected to anything live. A light started blinking at the top of the rack, and a message came up on the control terminal in the test lab.
Just then, across the large data centre floor, through at least 2 wire reinforced glass walls, a mainframe crashed.
They blamed me.