If ut ain't broke don't fix it
Especially if it's been unbroken for 25 years. The moment you touch anything like that is usually the moment the gates holding back the hordes of hell break and you are overrun with disaster.
Good Friday morning, and welcome to our weekly installment of On Call, where Reg readers share the tech support moments that made them want to rip their hair out in despair. This time, we meet "Geoff" and travel with him back to December 2010, to what sounds like one of the most frustrating client visits of all time. The …
This post has been deleted by its author
Many years ago, working on an OS as a naive and new support tech, I cam across some obscure code written in assembler that looked like the programmer was showing off and should be rewritten in the high level language that everything else was written in. What it did was commented for the call, but how it did it, wasnt documented at all.
i showed it to the manager and asked about rewriting " ah yes he said, no one really knows what that does, but it seems to work so so dont **** with it"
It is the dirtiest four letter word in systems..
I work in a job where I am occasionally asked to design or change various systems. Many is the time when I've been asked to make a "simple change" only to find that "simple change" needs hours (or days) of coding, then several days of testing.
Thankfully, we have introduced a "Request For Change" system which requires that any changes be documented and approved, preferably before being made (although the system does allow for approval to be sought retroactively in emergencies). It seems bureaucratic, but it's there to ensure changes are authorised and also to function as an audit trail if something goes wrong. It's being enforced at all management levels from directorate level. It also pisses off the users, which can also be a good thing.
"Quicker, cheaper, delivering in increments that may well have imperfections, and only if it's a small project of carefully scoped goals using your very best, experienced people given prompt access to resources and users ..."
... but the beancounters and clueless MBAs who infest the industry like guinea worms only hear the first two words.
Reminds me of a place I used to work in; "We live by the motto Live Fast and Break Things". Yeah, that might be great when you're writing a game for mobile devices. Not so great when you 're trying to persuade me to work with it and I'm the infrastructure guy.
I don't think the rest of the office would appreciate me taking down the network, switches, router, servers, just to see if I can make things better by messing with them. I prefer a stable setup for these kinds of things.
Anonymous because although the company I was working with then is no longer operating (yeah, quelle surprise!), I do get on with others who used to work there so I don't want to embarrass them.
"Not so great when you 're trying to persuade me to work with it and I'm the infrastructure guy."
They are talking about the product, not the infrastructure. The phrase "Build it, race it, break it, repeat." applies to the vehicle, not the toolbox. If you don't understand the difference, you're fired. If THEY don't understand the difference, keep an eye open for a new job and hope you find one before the inevitable happens.
There's a difference between a project you're trying to ship to your benighted customers for cash money vs a product that your STAFF actually use in-house for important things. I mean, eventually, your customers will get sick of the shite you're pumping out and go elsewhere, or maybe what you're trying to flog isn't that important.
Try explaining that difference to a manager who's been on 3 day Agile trainnig and fancies himself as a "scrum master".
I actually do believe in a phased approach to internal product delivery rather than a classic waterfall, but you need to understand exactly what those phases will deliver and explain that to your "customers".
Agreed. Same goes when visiting a friend/family member and they say "you know about computers - can you just take a look at..." which usually results in 3 hours of un-fun in the spare room disinfecting, reinstalling and updating a PC so rabid it should have been taken behind the shed and shot, whilst everyone else is downstairs merrily drinking tea & scoffing biscuits.
As I work in midrange systems and storage, so far I always got away with
1. Sorry, for all this consumer stuff stuff I am just a user as you. (Which is an obvious lie and will immediately answered with):
1b Yes, but please (add some honey here)
Which in turn causes my (so far unbroken) safety fallback to trigger
2. I only do systems I can not throw out of the window. Do you want me to test whether I might be resonsible for yours?
Kudos to my wife when she replied with a straight face whether I was joking: "Last time, he was not. Well, the netbook was broken anyway."
"It's a well established fact in my family that I fix Windows by installing Linux "
Other than pulling their data off first, that's 90% of my computer "support" these days. Mainly switching the language is simple, compared to switching a Windows install between English and Arabic.
"Just shrink the Windows partition to fit Linux and stick it somewhere such as /mnt/cdrive. LibreOffice will read all their Office files."
Oh, I wish!! Most of my relatives are into hobbies of one sort or another. Genealogy, crafting etc so moving them to a non-Windows system would be an even bigger job, if not downright impossible. File conversions, hardware drivers, apps to drive the hardware etc. is a complete non-starter. I can export Family Tree Maker data to Linux, but lots of data doesn't go because FTM allows for all sorts of data that none of the other formats will take. Cutting machines that look like printers but nothing in Linux will even see it let alone drive it etc.
In my fathers case it was also something like Family Tree Maker, so put on Ubuntu, stuffed it in to an XP VM and gave it no internet. He uses Linux for email/web and so far (5+ years) it has been good with no compromises or mysterious crapware appearing. A big advantage of the VM is you can have a couple of them if the old curddy applications don't play nicely together for whatever reason, and no hardware activation issues if you change to host PC hardware.
With relatives and many "friends" (the real friends I'll attempt to help out usually with them sitting there along with keeping refreshments flowing) I just tell the Win10 ones, "I don't know that one as I've not installed it". For the Win7-8 users it's "I've not kept up with that version.". $<deity> help me if they ever compare notes/excuses with each other.
"Not switched over to Linux yet*, but still running Win 7 at home and finding I can use the same excuse on anything after that with how much they've changed the interface"
Interesting experience on Win10 the other week on a relatives PC. I thought it best to make sure it had all the latest updates so went looking for the settings and decided to speed to just type "windows update" into the search box. No useful results, just "internet" results. Weird. So I changed the search to "update" and the first result is the link to "check for windows updates". Fscking strange!!
My experience: Usually run Windows 10 updates after an unplanned restart. Type "update" in the start menu, "Check for uodates" does not appear. Delete then re-type the last character, "Check for updates" manifests itself. I think it's a security measure to prevent accidentally loading the October 2018 release.
Even better, convert them to Linux. Me DearOldMum (mid-80s and computer incompetent) & Great Aunt (104 years young and computer illiterate) use a version of Slackware that I provided for them. Their support calls have dropped from several times per month for software issues with MS products, to once or twice a year for hardware issues. Nowadays, instead of having to visit several times per month to work on their computers, we visit a few times per month because we want to.
"Oh, it's Windows. I haven't touched one of those for years, sorry,"
This also works for the most painful Linux problems, especially when they're driver or audio related. "Oh, that's probably pulseaudio...is this the new Ubuntu UI? Sorry, I use ALSA and Jack on Devuan, have you tried searching Google?"
I had hoped that would be the case but I'm finding it's not working for me. It's been a couple of years now and I genuinely feel out of the loop with Windows but I still get,"Yeah but you know *something*, surely you can just take a look?"
So then it's,"What little experience I have with Windows 10 has been very frustrating, I'm liable to get wound up and make it worse." Which even people who know me seem to think is a joke, confusingly.
So it's,"*sigh*. Got beer?"
Same as it's always been with printers. No matter how much I protest that I'm crap with the things and they hate me it's,"pfft wouldn't hurt to just look?"
"you know about computers - can you just take a look at..."
That stopped a while back for me, thanks to a simple strategy - when asked "can you just take a look at...":
1) Protest strongly and repeatedly that you're a programmer and "don't do hardware/operating systems"
2) Eventually give in and reluctantly agree to take a look
3) Turn their computer into a very expensive doorstop
Word soon spread...
1) Protest strongly and repeatedly that you're a programmer and "don't do hardware/operating systems"
I did that to a friend once. I said to him, "you work in "transport", don't you, can you fix my car for me". He spluttered a bit and said he was a logistics dispatcher. So I said I fix hardware, why would he expect me to know how to fix windows. He got the point.
The inverse of this is worse ... When a friend/relative who knows nothing about computers tells you how to fix their machine according to some arcane lore they received from a friend who works at HP/Apple/Dell/TI/whatever. Their friends career path? Building maintenance ...
Trying to upgrade and/or replace longstanding legacy code is generally a passport to unending pain, grief and torment. I speak as one who is in the middle of doing exactly this, with a long-standing Linux set-up currently run by a one-of-a-kind homebrew config and provisioning system, even as a management-led replacement project runs in parallel to my replacement/upgrade project.
Did I mention that I am looking for alternative employment even as I speak?
Been there, done that. Back in the days of unloved, unlamented Windows Me.
We were invited to a friend's house for dinner (or supper, if you prefer). I was asked to take a quick look at their PC. I wound up spending 4 hours slaving over the thing, while everyone else enjoyed a lovely meal. And I do mean lovely. Said friend's wife was an excellent cook.
It was at that point I vowed, "Never again!". My stock response when asked what I do for a living is now, "I'm a software developer, and no... I WON'T fix your computer."
For family emergencies like this, there's the ThinkGeek "No, I Will Not Fix Your Computer" t-shirt, backed up by "no, I mean it"
And I'm in the "I'll convert it to Linux" camp, as it's far easier to ssh in from my phone when I'm 300 miles away on vacation and get "MAH COMPOOTER DIED" plus it's far less susceptible to the usual virus-laden emails from Nigeria.
You're all over thinking this. The correct answer to any "Can you just have a look at my computer?" problem is: "Sure! Here's my consultancy rate, how would you like to make payment?"
At the inevitable look of confusion and disgust, ask them if they will "just" do whatever their job is for you. For free. In their spare time.
Most people get the message, everyone else isn't worth bothering with.
Worst part is, many times you're not allowed to say no because the person in question is an elder or someone quite important in the clan, meaning any attempt to say no would be seen as a mortal insult which will be avenged even if you tried to go into hiding. Example, one with a sizeable estate, upon hearing an attempt at a denial, started talking about rewriting her will...
Family are gathered in the wood panelled room, a stuffy solicitor with bifocals on the end of his nose reads out the last will and testament. While the occasion is tinged with sadness at the passing of the relative, the family are also a little excited at getting a slice of the breakup of the estate, property and premium car.
It's now AC's turn...
"... and to AC, I know how you have always been there as a huge help with any technical issues I had, usually calling round as soon as I phone you that I'd had a problem. So to you, I leave my beloved 2010 Acer Aspire One netbook, the one that you liked fixing."
Same goes when visiting a friend/family member and they say "you know about computers - can you just take a look at..." which usually results in 3 hours of un-fun in the spare room disinfecting, reinstalling and updating a PC so rabid it should have been taken behind the shed and shot, whilst everyone else is downstairs merrily drinking tea & scoffing biscuits.
Simple, you say, sure, if you can (insert whatever they are able to do - weed my garden, mow my lawn, clean my bathroom, hoover, etc. etc.) for the same amount of time it will take for me to fix this. And, I will come back and fix it when convenient to me, not when I am supposed to be enjoying myself, or I WILL call you out to weed, hoover, etc. when YOU are supposed to be enjoying yourself. Deal?
I do Windows Server/IDAM, so the punters get told a) I'm almost as useless as they are with Win 10 and other MS workstation OSes and the cruft MS insists on jamming into them b) I require bribes, for the entire duration that I work on their virus-infested underspecced PoS and afterwards. The post-work bribes should consist of bottles of alcohol of a value exceeding $30 (AU or NZ), at least one per hour I've had to swear over their garbage.
The best one I had, though, was the mystery Windows PC that "didn't work most of the time" when they tried to convert part of the garage to a home office.
What's not working? (when people say "the PC", they mean Word, web browsing, or power is off)
- The internet. (yup)
How's it connecting to the internet?
Where is the wifi router?
- In the house.
So I connect my phone to their wifi and disable mobile data. Walk outside, nice steady connection, lovely. Into the garage, still fine. Close the *metal roller door* to the *metal garage*, whoops, no more "internet".
Luckily the wifi card in the PC tower was attached to its aerial (pointing at the *metal door*) via a metre-long RP-SMA pigtail. So I just had to move the PC to the top of the desk, open the window above it, and drop the end of the aerial out the window.
Very satisfying, especially since one of them had a brother working as a software dev earning about a gazillion bucks more than I did, who'd looked at it for an hour before giving up. Two excellent bottles of French wine bribery for 10 minutes work. I threw in the furniture moving for free.
Went to some musical instrument making workshop. Introduction included: do not use "just" (well: "nur noch"). So when we take a break for coffee or lunch or so, do not say "but I just only want to..." because that's the moment you are going to saw directly into your soundboard, undoing a few days of work
Also note the BOFH once observed that in a request it is a signal word for an unreasonable one...
In my early working life as a painter and decorator in the UK, the worst words you would hear from a potential client was ' I just want a clean up' or 'I just want to freshen the place up'.
That meant they wanted to pay peanuts and then complain about the lack of quality of the job and then try to pay even less.
Just should just be stricken from the language.
No houses but cars here. I built some drag race cars for a few folks and word got around. When told "I just want to go a bit faster" my typical response was: "Speed cost money, how fast can you afford to go?". They usually had to rethink their request. It was a hobby but if I had gone along with what they wanted, it would have been a full time job that would have lost money.
Ditto in electrical installations "Its just a switch I need adding how hard can it be"
Well very if there's no wiring anywhere near it, that there are no crawlspaces under floors and the house was originally wired in pyrotenax / MICC (trust me you don't want to go disturbing that if you don't have to, especially as very few still have the tools to work on it or recent experience using it (most use FP200/400/600 now as its a damned sight easierr to use.)
Damn, you made me look - I still have a set of Pyro pot crimpers and one of those nifty BICC ringing tools.
In the right hands, it was an art of electrical installation, I remember seeing a fairly nondescript church hall brought to life by the polished (not the bright orange or red covered stuff!) runs of perfectly formed Pyro - it really is a lost skill.
"In my twenty-or-so years commercial experience as a software developer, my biggest take-away is that "just" is the dirtiest four-letter word in the language of specifications"
Absolutely right! I actually have a rule I've been using for years now, to reprimand anyone using "the J word" in any sort of request or description of a requirement.
"the J word" will always bite you in the ass.
Not as bizarre as you might think. In the early 1980s I came across a piece of test equipment 15 metres long and weighing many tonnes. It was controlled by an early microcomputer, which interfaced to a cabinet full of thermionic valves, including a lot of KT88s in the driver stages.
If there isn't a blue glow you aren't getting the envelope hot enough.
(These ones were fan cooled, though.)
On a past technology note - thermionic valves switch off by the grid going negative. The interface used some rather unusual PNP silicon transistors to switch from 0-5V logic to 0 to -24V to drive the thermionic stages. All these intermediate technologies that have gone the way of the dodo in just a few years.
I was at a MEMS and Microelectronics presentation, and it was claimed that if you tried to implement an iPhone (4 at the time I think) using original valve technology, the resultant "device" would be the size of the Post Office tower...
Not sure how accurate but it served to illustrate the massive changes microelectronics have wrought.
While trying to understand some controller software for a video production facility I found an unnamed valve that was initiated once every 123 videos. I was just bored and trying to work out how the place ran and didnt really think about it until a month or two later when fixing some shit on the air lines on the conveyor I came across a cardboard box by the side of the line as it came out the 'other' side of a wall in an fairly well hidden part of the factory with easy access to a fire door opening onto the car park. Sure enough every 123 videos one the valve actuated and flipped a tape off the line into the box for delivery to the back of pub somewhere. As I worked the night shift and never interfaced with 'proper' management I mentioned it to the night shift controller who said he'd deal with it.
And I think that's how I got promoted to another unit!
A long time ago we were working on a bottling line at a whisky distillery in Scotland, which was a right pain as the line went in and out of the bonded area with security (annoyance anyway) between. You had to walk miles to get from where the line went into bond to get to the access door. The control lines ran the length (tens of meters) of the line inside the conveyor carcass with breakouts for the control ports, gates etc. Everything was sleeved and 'water'proof. All fine and dandy.
Sometime later we were called back for an extension and some additional work. When we came to tap into the control lines there seemed to be an extra sleeve not shown on the installation diagrams. After a lot of puzzling, and tedious back and forth it turned out not be an extra sleeve, but a hose pipe that some enterprising soul had installed to help the angels get their share.
I had to work on an HP system once, we had a terminal in the customer's office, which was connected at 1200 baud to a serial multiplexer, which was in turn connected to the local Ethernet, which went through a modem to the data center, back over Ethernet to another serial multiplexer and into the HP mini computer...
The thing only had a line editor that made MS-DOS edlin look like a state of the art WYSIWYG editor!
Want to delete the 25th charcter on the line? 24 x space, "d", Enter.
Insert something? Space out to where you want to insert, i, text, Enter.
And so on. The only hitch was that all the speed changes caused major delays and echoing problems. I had to edit a report program that pulled data out of a database. Certain columns of each line mean certain things, so very particular about the correct syntax.
The system was so slow that it was taking minutes sometimes to display 1 character entered (data coming from the mini was fast enough, it was the roundtrip stuff that was slow). I was bashing away 3 or 4 lines of edits, using an imaginary line in my head to work out the correct number of spaces etc. for editing the lines, then going away and doing something else, then coming back half an hour later to see if my edits had gone through, and more important, whether I had made any mistakes...
The thing only had a line editor that made MS-DOS edlin look like a state of the art WYSIWYG editor!
Okay, so it had Vi.
Want to delete the 25th charcter on the line? 24 x space, "d", Enter.
Insert something? Space out to where you want to insert, i, text, Enter.
Yeah, you said. It had Vi.
/me scuttles off to find a bunker...
"VIM is actually the chosen one"
Vim? Beware of cheap imitations. My first contact with it was enough to teach me never to trust it.
I fired it up to tidy up a file that came from an MS box including removing the ^Ms off of the line endings. I discovered it had been set up to "helpfully" hide them. If vim is prepared to hide those what else might the lying little bastard choose to hide? Any time I set up a Linux box one of the first steps it to replace vim with nvi.
VIM works perfectly well as a vi variation.
However, a so-called "poweruser" that doesn't check the .rc for local variations of a very customizable editor isn't going to get admin privs on any system that I maintain. Operator error isn't the fault of the program. Or, if you prefer, it's a poor craftsman who blames his tools.
With that said, there's nowt wrong with nvi. Install both. And stevie. And elvis. Etc.
However, a so-called "poweruser" that doesn't check the .rc for local variations of a very customizable editor isn't going to get admin privs on any system that I maintain.
Conversely something describing itself as "improved" isn't going to get installed on a system I maintain, not if there's an alternative. In a third of a century of use I've never felt vi to be lacking in those "improvements" so I regard them as featuritis. Obviously we choose to differ on this.
To be fair to vim this is the only instance that I came across so egregiously set up out of the box. But it's like a restaurant; if you had a dose of food poisoning the first time you ate there even if it had a constellation of Michelin stars you'd be likely to feel a bit queasy going in there again.
vi is only a line editor in ex mode.
In normal mode if the terminal uses escape sequences delays like those described bugger up the timing so badly the escapes get misinterpreted. In particular you get the situation where an escape sequence with a tilde in it is strung out so the tilde is seen as a character in its own right and flips the case of whatever character the cursor's on. So, no, he wouldn't have been using vi. Maybe its alter ego ex or ed. ANd that assumes the server was running HP-UX; I've not idea what editor's on the non-Unix HP jobs.
>Okay, so it had Vi.
>Want to delete the 25th charcter on the line? 24 x space, "d", Enter.
>Insert something? Space out to where you want to insert, i, text, Enter.
>Yeah, you said. It had Vi.
>/me scuttles off to find a bunker...
Scuttling off to a dungeon might be more appropriate. Since it was a line editor he did not have "vi" he had "ed".
Vi - Visual Interface was originally merely an interface to the standard Unix editor "ed".
Very good spotting though, well done.
Editor designed for teletype.
Had teletype commands in the Univac line editor, but you had to find 'em and most people didn't get past the Big Four (insert, delete, change and print).
The Univac teletype commands were great for precision mods of complex lines with minimum typing.
If I've learned one thing over the years of writing software for myself, reading such horror stories, and dealing with companies who do exactly this, it's this:
- Unless you have a complete, working, development suite, which can run on modern hardware (even in a virtualised environment), with the complete, working, original, version-controlled source code, which compiles immediately, to the target platform, and whose output is then directly related to the binary that is actually used on the system (i.e. you have the source code to version 2 which, when you compile it, outputs an identical binary to the v2 binary that the company is actually running) then it's just not going to be worth playing with it.
The same way that people expect me to somehow "fix" Microsoft's software bugs for them: without the above, I can't do anything about it but what the software supplier themselves give me / tell me to do.
I'm not scared of tinkering with software to get it working or make it work the way I want. I've banged on the Linux kernel to do some very specialised custom stuff on the backend (only ever run on one machine, but it did some really useful magic for that one purpose). I've hacked around the code for projects like Hylafax and Samba to make it do things I needed it to do. I coded addons to access control systems that didn't have exposed APIs, even, so that we could do things we wanted. I've saved companies thousands doing so. But I leave behind the above state to anyone who might follow. Chances are that it will NEVER be taken over by someone who can even understand what I've left them, but at least if they are capable, it's there for them. Usually, though, it just gets ripped out the second I leave (and the people who allow me to do such things know that, but if it saves them a few years of licensing, and also proves to their software suppliers that what they want *is* possible if they pull their finger out, they'll happily let me do it) and replaced with the same inadequate bog-standard, expensive system.
This is the primary driver for why businesses want off-the-shelf, industry-standard pieces of closed-source software from an active company over anything else. They don't care... they don't want to care... it's someone else's problem, even 25 years down the line.
But I've learned that if you want a custom system... you need the above at minimum. Even the code is probably going to be utter trash and without version control you'll have no idea what the hell was changed or what it was trying to achieve. But at least you stand half-a-chance of getting something working.
Hell, I avoid having to re-setup my own development environments, because of the work invested in getting them going properly and producing the right type of files with the right embedded binary portions and so on. Let alone the code itself.
One of the reasons that I like open-source systems... I at least stand half-a-chance of a configure;make;make install actually reproducing the binary on my system, and then from there I can start making changes to the version-controlled code that underpins the binary I'm actually using.
"Unless you have a complete, working, development suite, which can run on modern hardware (even in a virtualised environment), ... then it's just not going to be worth playing with it."
Unfortunately, even if you don't have that when there's a few million quids-worth of industrial plant paying lots of wages involved somebody has to play with it. It's what's called legacy. It might be ancient, creaky and undocumented but it's what brings in the money to pay for all the new shiny. Despise it at your peril.
And that's why people moan about SCADA systems being vulnerable, and why its probably not wise to expose them to the internet.
They are designed by people that are engineers in the more traditional sense, not software people. Its gets developed to the point where it just about works as a sort of permanent proof on concept in most cases.
The environment it is installed in is also unsafe, inhospitable, built to grandparents safety specifications, and been modified and bodged for years. Often using second hand equipment doing a job it was never meant to do when it was designed.
Never actually broke any of these personally, but had to work around some, and attempt implementations of new stuff while trying not to disturb the existing rusting dystopia that my solutions had to work in.
I think the guy in the article got of quite lightly.
"They are designed by people that are engineers in the more traditional sense, not software people." Or just as likely (more so in my experience), software peple who haven't got a clue about the real world. Unfortunately some of the software people I was dealing with with the system I worked on were the main dev engineers. Common problem: 12 controller cards in a cardfile, 11 are operative, the 12th was a standby failover card. Software problem on one of the 11? Fails over, then takes the backup card down too, and with it, the plant. Or the asynchronous "set this valve to X position" call, which then allows the next line of code to execute without the valve yet being as required. Had to add in a synchronous call function into the basic opsys - after 2 years of this system being in existence. How did the deve engs not know the set statement had to complete and get feedback from the valve position. In a major control company. In a Process Engineering environment. Like they had been working in for 20 years+
If a system is not documented the only reason to be touching it is for complete removal. In this case I'd also have some serious questions about the safety implications. What if this system addition changes edge cases never foreseen, tested or mitigated? What effect does it have on timing?
To me it sounds like a project that should have not been touched with a 10 foot barge pole, since it there was no original documentation, the current system performance/operation wasn't documented either and the factory team responsible for the installation of the valve seems to be completely and utterly incapable of doing their job.
@El Reg said: "Apart from anything else, all drawings and notes had been put in their possession, as per the original contract, and guess what? They'd lost them," said Geoff. (...) "First, the control sequence was not the same as what we'd been told, so we had to rewrite the program on-site to take this into account."
Been there, done that. You have to assume this is the state of affairs in any industrial retrofit project you go into. It seems to be almost standard practice in industry. You need to analyse the system and look for this problem well before the scheduled date of the retrofit so you aren't trying to deal with it on the fly.
@El Reg said: "We eventually realised the guy had discovered a way of bulk updating large blocks of data. He'd used this enthusiastically, blissfully unaware that it also corrupted variables in the 'gaps' that he wasn't using – but we were trying to."
Let me guess, it was a Siemens S5, right? There are instructions which directly overwrite absolute memory addresses. You can not only overwrite data, you can actually overwrite code as well. How do I know this? From having to debug a program written by one of the biggest equipment companies in their industry when everyone was convinced it was a hardware problem while it smelled like a software bug to me.
The problems with S5s was that as well as ladder they gave you access to a lower level instruction set that had powerful instructions comparable to assembly language, but which ran in an interpreter (except for some models which had a custom logic coprocessor running M5 instruction code, but that's another story). This looked sooo seductive to bored full time PLC programmers who would find "clever" ways of doing things instead of using less interesting but more readable methods. The more formal education the programmers had, the more they were inclined to do things like this.
The Siemens S7 series which replaced the S5s took away some of the more dangerous instructions, but added even more pointlessly complex low level ways of doing stuff. The result was of course loads of unreadable S7 code out there, which even the author struggles to understand. This more or less defeats the whole purpose of having a PLC in the first place, as the software is intended to be able to be understandable by non-programmer electricians. I think that some people see this unnecessary complexity as a form of job security.
It is comparatively easy to solve a complex problem with a complex solution. It seems to take least 5 to 10 years working in the field on a broad range of industrial problems before someone gains enough experience to come up with simple solutions to complex problems. Some people never seem to reach that point.
The best PLC programmers that I have met tend to be older electricians, while the worst seem to be younger electrical engineers. The former tend to think about how something is supposed to be maintained after they've walked out the door, while the latter tend to have trouble imagining that something they worked on could ever go wrong.
This ^^^ so much this. Complexity and complication are orthogonal yet invariably a complex problem is given a complicated solution when a bit more thought would produce a straightforward one. Anyone can make an over-complicated solution, but it takes a lot of effort or experience to make a solution look easy and obvious.
All sounds entirely normal commissioning to me. I am still suffering the physical-after effects of when the valve that moved, that shouldn't have, was a 36" diameter gas line valve from one of the major offshore high pressure (70+ bar) transport lines coming to the beach. And why is this noteworthy? Cos it was in 1988.
I worked for company made valve actuators which are the fancy motors that turn pipeline type valves and they had recently finished their new product. The first one installed in Australia at the Longford gas plant on 24 September 1998. I know the day because there is a wikipedia page about the explosion the next day. While the device had nothing to do with the fire, the local news paper had a nice front page photo of the damage to the plant so I sent that back to company with a nice note saying that it did work as advertised. The device had been in debug mode and had recorded quite a bits of data, some of which was used to figure out just what had happened the other side of the plant. At least the company had a nice photo of one of their test sites.
One of our customers actually took an ancient sun box, running a critical system, and virtualised it into one of their parent company’s data centres. They still pay us for development work, from time to time. Everyone’s heart sinks when asked to work on Sun (especially old Sun - we’ve all adjusted quite nicely to Linux, and wouldn’t go back), but it’s usually good money/margin. Of course it helps we’ve retained older skills in house, while re-tooling for Linux, virtualisation and the cloud.
With 8 service centers world-wide to cover the huge number of F1s made (64 road cars in total), I can see why one might want to do one's own tune-up and oil change ... The wait at the dealer might be on the order of seconds! You'd think that with many times more factory authorized technicians than actual automobiles they'd be able to cut the wait time down a trifle ... talk about poor customer service!
Sarcasm aside, a bit of a reality check: I know four people who own F1s. All four cars are garage queens, not a one of them has been driven ten miles by the owner. They are not really automotive machinery, they are status symbols and investments ... the haberdashery of the hyper-wealthy. I seriously doubt any of the owners I know has ever given a thought to servicing their toy.
To give you an idea of how much these things are more about hype and marketing than reality, "McLaren San Francisco" isn't located anywhere near San Francisco ... it's located 35 miles South, in Palo Alto. Seems that these days SF has more cachet than lowly Palo Alto, home of the gauche Tesla.
It's interesting to note that "McLaren San Francisco" is located directly next door to the Palo Alto Tesla salesroom...If a Telsa does not have enough cachet for you, you can pop next door and get a McLaren.
(I wonder whether Elon Musk still has the McLaren he bunged up several years ago, showing off to a friend on San Hill Road.)
Last time I looked, Volvo was between McLaren & Tesla ... Quite convenient, that. Three brands I wouldn't touch with the proverbial barge pole, all in one easy to ignore location. (Well, once off of 280 I usually take Arastradero instead of Page Mill when I visit my parents, but it's easy enough to drive on by.)
I was working as a contractor for Siemens and by some unfortunate luck I got pulled to work Christmas Technical Support. Not really a big problem triple time and all that, but I had a new crafty co worker that liked to skive off. So the place is pretty much dead, feet on the desk watching youtube, and my co worker decides to forward all the phones to his mobile and goes down for a pint at the local. He doesn't tell me this. A couple of hours later he doesn't reappear and nobody is calling in. Didn't see anything different. Well while this was going on we had a DDOS incident and nobody could get on the website (it is a very much used one and has a million quid draw on Christmas Day) so my boss calls me and asks why all the calls are going to a swearing joke voice message and the mailbox is full. What was funnier that the PBX was down too. I got the web server and PBX back up in record time. My co-worker turned out to be a senior manager, and got the sack. I didn't have to work Christmas again, but everyone else had to in my department. Fun times.
Biting the hand that feeds IT © 1998–2020