Reply to post:

Exsparko-destructus! What happens when wand waving meets extremely poor wiring

Nifty Silver badge

I worked for a company that made process control systems. It was in the days of film cameras with a big electronic flash on top. The installation technician would start up the system for the first time with the processor cabinet doors open. Check that all processor lights were blinking rhythmically as expected. Check the SCADA graphics (and have a good listen to the factory noises) to see that all valves were in the correct position and motors off so that the system was 'at rest'. Now was the time, before locking up the cabinet, to take the photo that went into the project album. These were used by the salesmen or for reference before service techs went to a site.

Flash, then bang and wallop. All processor lights now frozen and valves and motors could be heard activating at random across the factory. Turns out the EMP from the cameras flash was enough to upset the processors so they crashed. One processor crashing was enough to corrupt the shared RAM so the others would crash. In the process, I/O sent signals that caused plant activations*. Normally the risk of this happening even in a lightning storm existed but was low, because the steel cabinet doors are kept locked.

Previous photography had been done the same way without crashes but there must have been a probability that every Nth photography flash would trigger a crash. The procedure was modified after that to ensure that the rack was powered down in the presence of a flash gun.

*For those worried about the implication that factory fires/explosions can occur if automation goes doolally, there were additional direct safety interlocks that override to (theoretically) prevent overfilling/over pressure, overheating, etc. But things can still require a lot of cleaning up and checking after an incident.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon