Security needs to learn from the aviation biz to avoid crashing The security industry needs to take a leaf from the manual of an industry where smart incident response is literally life and death, if it is to fix systemic problems. In a presentation at the Black Hat security conference in Las Vegas Tarah Wheeler, a senior fellow to the US Council on Foreign Relations and founder of …
While C-level exec's aren't personally liable for security breaches the problem is not going to be fixed.
The only other way I can see going forward is if governments vote with their wallets either penalizing companies with poor security records through not buying their products / services, or simply through linking the corporate tax rate to security.
Well the government keeps going back to the same large IT companies time and time again, no matter how much of an unmitigated disaster the previous projects have been, so I doubt if mere security issues are going to persuade them to ditch these serial cock-up merchants.
> Well the government keeps going back to the same large IT companies time and time again
Remember it's not "the government" though. It's someone in the civil service who works under the principle that "none has ever been fired for buying from X". Gluteal protection and all that.
It's worse than that, in many instances they aren't permitted to choose and have to do a tender which essentially gives it to the lowest bidder 99.9% of the time.
Recently took part in a national procurement worth millions, when you look at weighting, cyber, info and infrastructure security was worth less than 2% of scoring. So basically worthless.
And as well as that, chances are that the team responsible for the specification and procurement are well outgunned by those tendering.
Put simply, if you are the sort of person we need to be driving these projects, you probably aren't interested in working for civil service pay. Lets face it, given the choice of working in a system where political dogma limits your pay to well below inflation (yet again) and senior politicians seem to be queuing up to slag you off, or joining one of the "big four" for much much more money and status - which are you going to choose ?
The very same Dido Harding who was CEO of TalkTalk when this shit storm happened:
https://www.bbc.co.uk/news/uk-england-stoke-staffordshire-46264327
https://www.itpro.co.uk/security/24136/talktalk-hack-two-men-plead-guilty-to-talktalk-hack
https://ico.org.uk/about-the-ico/media-centre/talktalk-cyber-attack-how-the-ico-investigation-unfolded/
The hack was in 2015 and at the time TalkTalk didn't even use HTTPS on their customer login page. Despite customers having told TalkTalk about it many many times.
Apparently this made her an ideal choice to run the governments COVID Test & Trace IT project....
This pretty much applies to most tech. You buy a license to use software with no guarantee it actually works. You buy hardware and it mostly does what you want but then when some flaw is found huge compromises are needed.
Then we have all AI & Self-Driving stuff that is exactly the same.
There is simply no accountability what so ever because the culture is that any development errors, operational flaws or straight mis-representation hare "accepted" and just "happen".
I used to do a lot of IT work for lawyers. I had a great conversation with one of my lawyer clients about the IT industry one day. We were fighting with a software vendor who's product was totally broken, and they had no intention to fix it, and did not want to issue a refund. His comment to me was "How does an industry continue to function when outright fraud is a normal everyday business practice?" I asked him "What if I took all of your computers and tossed them into the dumpster out back, and replaced them with mechanical typewriters? How productive would your office be?"
I remember a friend who was learning to fly and taking a training course for about a year before he finally passed the test and was allowed to fly with regular people ... I was the first non-pilot sitting next to him as we flew over Melbourne, and as we were heading into the runway after an hour he told me what he'd been taught;
"What's a good landing? It's one that you can walk away from."
But thankfully (though far too late), that is starting to unravel.
At the moment things are concentrated on compensation by way to overturning the conviction and paying compensation. I would be very very surprised if quietly in the background, all the evidence that's now coming to light won't be used against those who committed outright fraud and perjury. There's more popcorn quaffing scope in this story yet.
YET
Now the cat is out of the bag, there is a lot of information coming to light about who knew what and who ordered what. As I said, at the moment things seem to be working on dealing with the sub-postmasters who were wronged - but with all this evidence that's coming to light, my feeling is that things will change.
Perhaps TPTB might try and wriggle out of prosecutions, but I suspect there's a big enough group of wronged sub-postmasters, and people willing to take on the challenge, that private prosecutions could be in order.
But right now, it makes sense to sit back and let some of the detail emerge - and hence evidence of wrongdoing be shown - before starting that process off.
And when it does, I expect some people to be falling over themselves to testify - in order to save their own skins. A key things will be the techies involved who will certainly not want to be the scape goats for manglement - some have already testified that they told manglement about the problems and were forced to hush it up (as in, "speak up and never work in the industry again", sort of encouragement to keep quiet). Senior Fujitsu people will be keen to push as much blame as they can onto the Post Office, senior Post Office (of the day) people will be keen to push the blame onto Fujitsu - we've already seen Post Office people claim that they didn't know, while Fujitsu people have come up with evidence that they did. Similarly, senior people will be keen to say that juniors kept the facts from them, while those juniors have been happy to prove otherwise.
Given what's at stake, such proceedings can take a long time to grind their way through the system.
"Perhaps TPTB might try and wriggle out of prosecutions"
Several people DIED as a result of their actions and they seem to be working on the principle that if they drag it out long enough, most of the rest will too
Post Office needs to be on the hook for corporate manslaughter by harrassment - and if it's proven that senior manglement did know of the issues then they need to face personal culpability
I agree completely.
The only problem is that, when an airplane crashes, deaths are involved and the number of specialists that analyze the issues are limited and highly experienced.
In the world of computing, a company cannot lock down a server for a full forensic examination, it needs to continue making money and it didn't foresee the expenses for a second server with the same configuration to pick up where the hacked one fell off.
Computer security is hard because you can be hacked without knowing it. If a plane has a problem, the pilot will find out and report it, and there's every chance the problem will be corrected.
Plus, in computing the experts are not always that experienced and they don't have the same moral incentive to find all the truth. So, 35 million records were downloaded ? Yeah, but no one died. It is quite possible that some people will be inconvenienced, some of them severely, but no one died.
I agree with the spirit of the idea, but if the industry took security seriously we'd already know.
Still, it doesn't do any harm to speak about it.
> In the world of computing, a company cannot lock down a server for a full forensic examination, it needs to continue making money and it didn't foresee the expenses for a second server with the same configuration to pick up where the hacked one fell off.
In the world of aviation when a plane crashes they can ground the entire fleet (basically worldwide) of that model, and tough luck for the airlines affected. Having to shut down a server and not being able to make money? Why not?
Err, not really. Look what happened to the 737 MAX, they were grounded.
Just imagine what would happen if a piece of software, hardware or OS had to be "grounded" because of a major security failing.
That would very rapidly change the situation because the billion $ companies that are creaming off the huge profits would suddenly find that their monopoly and business practices were no longer viable. This is even more effective in this age with everything on subscription.
It would be tough, even disastrous but it would force change.
Certainly in the arena of autonomous operation (think self driving) and AI decision making, both relatively new, it would have a huge change in the culture of the products for the better.
How about all Teslas, or the vehicles Waymo are using being prevented from driving on the public road under any conditions due to a software bug. It would very rapidly get rid of the culture of using the public as beta testers or as the environment to test your beta (alpha?) software & hardware.
This post has been deleted by its author
Most EULA are invalid in law. Certainly, if tested in the UK by a consumer (rather than a business) then a lot of clauses would be automatically invalid and unenforceable - c.f. Unfair Terms in Consumer Contract Regulations (UTCCR). That is why no case has ever made it to court - the vendor has always settled out of court as they do not want a precedent setting, much better for the proles to not know their rights.
Also, technically, any EULA that is not printed in full on the outside of the packaging is completely unenforceable - simply because for a contract to be formed, each side must know the terms of the contract before they enter into it. Therefore, if you buy some shrink-wrapped software, and don't like the EULA when it's presented during installation, you are entitled to take it back to the retailer for a full refund (contrary to their claims "but you've broken the seal").
With business to business transactions things are a bit murkier. There is a concept in contract law that a contract must be formed by a "meeting of minds". If there's a situation where you have the choice of accepting a contract or not being able to run a business, then the contract is open to challenge if it's very one-sided. With software that would be a grey area, because I don't think it would be possible to say that it's "impossible" to run a business without (say) Office 365.
Actually, it would never happen.
If this sort of thing came to be, yes, software development would rapidly improve and things would get better.
However...
Firstly, it might deal a brutal blow to open source because who accepts liability there? And who is going to perform all the testing? Especially if people expect it to continue being free?
And secondly, software packages would become prohibitively expensive to cover the costs of all of the extra testing. And possibly a surcharge stuck on top for when they do get sued.
This concept isn't workable with our current state of software development and sales. Instead we've actually managed to go backwards where it seems "the minimal that works" can be rolled out the door, with rolling updates for 'n' days/weeks/months/years to cover bugs found.
And secondly, software packages would become prohibitively expensive to cover the costs of all of the extra testing. And possibly a surcharge stuck on top for when they do get sued.
This.
It IS possible to buy "safe" software, that's been developed according to rigorous methods and with oodles of formal testing. But I don't think many of us would be able to afford to buy it.
And that's the trade off - we get a lot of "good" software for, generally speaking, not a lot of money. But perhaps things are a bit too one sided as things stand - far cheaper to let the users find the bugs, and it shouldn't be.
True. Though the FAA is partly to blame. They accepted Boeing's lies that the 737 Max was "the same" as earlier variants of the 737. That meant Boeing got to self-certify it instead of going through the full FAA certification needed for a new aircraft.
Let's follow that chain of blame a bit further. FAA was forced to accept Boeing's statements that he 737 Max was "the same" as earlier variants of the 737 because of repetitive funding cuts by Congress. When is Congress even going to apologize for killing so many civilians through negligence?
> In the world of computing, a company cannot lock down a server for a full forensic examination, it needs to continue making money and it didn't foresee the expenses for a second server with the same configuration to pick up where the hacked one fell off.
A company can be legally compelled to lock down a server for a full forensic examination, or more likely have the server siezed by law enforcement in a dawn raid on the data centre. And even in the best of times hardware can fail suddenly and unexpectedly.
If a company does not have redundancy or a disaster recovery plan for these well-known risk factors, then the senior management is negligent, potentially criminally so.
more likely have the server siezed by law enforcement in a dawn raid on the data centre. And even in the best of times hardware can fail suddenly and unexpectedly.
Usually while the authorities are flashing their badges & warrants at reception.
https://www.theregister.com/2022/07/11/uber_leak/
"Usually while the authorities are flashing their badges & warrants at reception"
Which is why in a lot of cases, once the authorities are coming in the door, any attempt by reception staff to touch a phone or otherwise alert anyone will result in them being piled onto VERY hard (interference with court bailiffs executing warrants is a serious offence, it goes well beyond "obstruction" charges)
"Computer security is hard because you can be hacked without knowing it."
But you can find out how to make yourself much harder to 'hack', using the same approach as is applied to air crash investigation, except that it's applied pre-emptively. It consists of working down from a threat scenario to increasing levels of detail until the problem is defined to the point where (in the infosec context) control is lost on any path. It was notably outlined by Bruce Schneier in his article Attack Trees in the December 1999 of Dr. Dobb's Journal, but it goes much further back in principle - right back to he 1960s when it was used for modelling accident scenarios for the Minuteman missile programme. What you finish up with is a model of the problem that can be traced to root causes.
Although Schneier included the concept of logical relationships between the nodes of the tree, he didn't take that further to calculate likelihoods, but that is actually quite simple to do, and well worth the effort.
The only problem is that, when an airplane crashes, deaths are involved and the number of specialists that analyze the issues are limited and highly experienced.
Also, compared to software security breaches, airplane crashes are rare. Because of that disparity in numbers, airplane accident investigation boards (usually a government/government-adjacent body) can be staffed with far fewer investigators while still being sufficiently effective, than a country-wide CERT team responsible for, or at least overseeing, security breach investigations.
While trying to learn from the reasons why incidents happened and define ways to fix and/or mitigate those attack vectors and publish them is nice it still won't get the IT industry to work in any way close to what happens in aviation.
If if were similar you'd have to file your intent to send a specific amount of data from your ip address using a specific protocol to a defined destination ("flight plan") and get clearance from your provider before opening a connection.
Before being allowed to connect to the net you'd have show you have a security concept and your system is handled by a certified administrator.
Government accredited network traffic control would monitor traffic and block unannounced traffic and revoke your certification to use the network if the traffic originated from your address...
The way it is now comparing networking connections in IT and IT security in general to the aviation industry is comparing apples and oranges.
> "Until someone has to go to jail for doing it wrong the teeth are not going to be the same,"she pointed out.
This is very much how we *don't* work.
I think the lady fails to appreciate the importance of a mostly blame free culture and way of doing things. You can hardly get people to open up and talk candidly about their mistakes if they're going to be punished for screwing up. Except in clear cases of wilful negligence we just don't do that. We just want to know what happened, learn from it, and avoid repeating the same mistake.
At the same time, there are sadly political elements that come into play in any complex investigation, but that's another story.
Yeah, she's got it wrong. Incident investigation in the air industry doesn't assign blame to individuals, and that's fundamental to how it works. It figures out what went wrong and how to fix it so that it doesn't happen again, recommending training, procedural or engineering changes without blame. The lack of blame is what makes it work, allowing people to honestly open up without fear. There may be criminal proceedings after an incident, but it's a totally separate process.
It's such a good process that is repeatedly cited as a way to fix problems in other industries, particularly medicine for which your think it would be a great fit, but it never works. It needs a total culture change so that individuals know that they won't be scapegoated for failure or punished for whistle blowing. It never happens. It always falls apart when someone in authority wants a quick, easy answer and then someone lowly carries the can.
As someone that works in IT Security, and is also a licensed pilot, I would somewhat disagree that the NTSB/Aviation Industry doesn't assign blame to individuals. I have read many NTSB crash reports over the years. There is a tendency to blame "pilot error" when another reason cannot be found. It is sort-of the default conclusion, unless something else can be discovered. Granted, pilot error is a common cause of accidents and incidents even as the industry tries automate the pilot out of the equation. We are now hitting the other side of the curve where automation is causing the accidents (737-MAX).
> I would somewhat disagree that the NTSB/Aviation Industry doesn't assign blame to individuals.
We don't assign blame *at all*, period.
Saying that pilot error was a contributory cause to an event is *not* assigning blame. We know that pilots are fallible and making mistakes is part of what the system is designed to try and cope with.
> a licensed pilot
Yeah most of our pilots are licenced here in Europe too :)
> Incident investigation in the air industry
It's not just incident investigation though, but a whole culture. You're encouraged to own up and usually protected from reprisal.
As everywhere else, there's a whole spectrum of how companies approach this in practice, but I would say that the general culture is pretty good.
Agreed. She may be intelligent, but she's not as smart as she thinks she is. If she had watched any of the airline pilots' youtube channels (and there are quite a few) where they discuss accidents, crashes and other mishaps, she'd hear, in nearly every video, about changes to processes, procedures, parts, ad infinitum, but only rarely about criminal charges being laid against the pilots, air crews, maintenance staff, etc. The reason, as stated above, is to fix problems, not assign blame and punish.
Why is it with other forms of engineering you can see when people build crap upon crap, yet we laud software that is bloated crap built on crap, like say Windows?
And why is it that the software has to put up with fixes for dodgy hardware? And why don't we punish Intel for its sins?
I don't think that crashing is the problem. The problem is that there is too much stuff that doesn't work upon which others rely on. If there was a case for removing complexity and improving quality I am all for that.
The problem is that there is too much stuff that doesn't work upon which others rely on.
Pretty much this plus a demand for yet more stuff (more complicated stuff in my experience, due in no small part to the need for security) and limited resources to create it. The world has been suffering from a shortage of software developers for decades and it's getting worse. We're not replacing the talent that is leaving(*). If we can't keep up with the current laissez fair software development attitude how are we going to implement tighter controls and increased oversight?
(*)On an unrelated note I'm now half a year from dropping to a four day week :)
When a plane crashes, the ad hoc organism searches for the reasons and then communicates the findings.
The 'communication' parts lacks too often with cybersecurity. The reasons explaining how an attack was successful are not communicated enough to anyone interested. Too often they are even kept secret. It's the opposite of what should be done. If we want that the whole world learns from experience, then it must be made public
The 'communication' parts lacks too often with cybersecurity. The reasons explaining how an attack was successful are not communicated enough to anyone interested. Too often they are even kept secret.
Well there are things like CERT that regularly publishes discovered issues, and when something "new" comes along - such as rowhammer or speculative execution exploitations - they get quite a bit of coverage.
Sadly, too many problems have very common design/implementation flaws - off by one errors, use after free, etc. Things that have been known about for decades, and with known ways of avoiding them or detecting them - but still many people fail to adopt these techniques or adjust their design/implementation approaches to make them more robust in preventing these errors (or enhancing early detection of them).
Whilst more could be done in analysing root causes, I fear that the general software industry has failed to take on board the existing knowledge-base or adopt the techniques that have been developed [formal/mechanical analysis of software is vastly improved since I first learned about it in the 1980's] - and that in itself is an major ongoing problem.
This post has been deleted by its author
when an aircraft crashes professional investigators spend time going over the incident to backtrace what exactly went wrong and why.
Who is going to pay for the investigation?
Who houses the investigation for each country or the world.
Where are the regulations demanding that systems are configured in such a way to prevent a tragedy.
Who is going afterward actor's. That intentionally cause harm to a system. And it's organization.
Some of you are pilots out there. Didn't you have to prove your skills to become a pilot? Don 't you have a license to fly?
Who is going to control computer administration and configuration. Who is going to pay administrators for these acquired skills and certifications.
Yes this is more than a can of worms.
I remember, many years ago, some states were talking about licensing IT people like they do with Professional Engineers. Nothing ever came out of it.
I could think of a lot of pros to it, but as with anything the government has their hands in, I can think of a few cons too. When little johnny goes over to grandmas house to help her with her PC, is he "practicing IT without a license"?
It's also the computational complexity of software states explodes in relation to the subset of components backed by mechanical linkages in an airplane. The most expensive part of the plane is indeed the software but it's designed from a very different perspective than user facing software that is generally required to be responsive to a variety of trained and less (or non) trained users. The possible states are much more limited, because firmware and software people working in aviation know that complexity will kill you, whereas in networking and productivity software the mantra is continual engineering, reacting to the bugs that are currently failing some subset of users. And once those bugs are fixed, the next bugs inevitably include bugs introduced by those so called fixes (in many many software teams) as well as some more subtle bugs and new bugs introduced by some unforseen assumption changing or new features demanded
It's not supposed to be comforting. It's supposed to be accurate. When something bad happens, the comforting thing might be to blame everyone, or shallow assurances that it won't happen again, but neither are the correct way to deal with a situation. The correct way is to really understand what happened and how it can be recovered from.
Sometimes, the best approach is not to blame anyone or to give consequences lighter than those harmed might want. This isn't comforting to them, but if we allowed emotion to dictate what happens to someone who is only partially at fault, we'd have a lot more feuding. When there is blame to go around, it's important that the people truly responsible get it, or we'll just get a bunch of angry people unjustly punished for something that's not their fault and people who are at fault happily proceeding to their next mistake.
I haven't watched the video yet, but I hope this little go to jail tidbit is taken out of context. Or something, because not sending people to jail for mistakes is a huge reason the aviation industry is as safe as it is today (that and the crew resource thing). The ASRS database being hosted by NASA.
Threats of being fired or going to jail cause more coverups, not more safety. This is common knowledge level stuff at this point.
I spent a quarter-century in corporate aviation maintenance, avionics and electrical specifically. I then moved into IT and was absolutely appalled at the practices. I still am 20 years later.
One "threat of jail" that actually worked was Sarbanes-Oxley in the US. SarBox had the threat of jail for the CEO and CFO.
Due to our fiscal year end date we were in the very first group that had to comply. The CEO and CFO were in learning mode a lot. A lot of sloppiness was corrected because of the threat of jail. The same thing was experienced when I worked for a bank in IT security years later.
In aviation, the way cockpit voice recorders and flight data recorders got the blessing of the airline pilot union was a federal law guaranteeing that neither could be used in enforcement actions.
The ISACS in the US are good for info sharing but sharing needs to lead to learning and too often companies do not care until they get smacked upside the head by an incident.
The onus definitely needs to be at the CFO/CEO level.
That's where the corporate culture is set, financial decisions on whether to spend on quality or divert to dividend are made.
Everyone else is just dancing to the CEO/CFO piper's tune.
The aviation industry runs on paperwork in the US. It takes ages to type certify new components, let alone new planes, and everything that's done to them, no matter how trivial, is documented. (Its why flying cars never took off -- the paperwork resulting from that ding you got in the supermarket parking lot....)
You can design software and write systems like this but you won't be able to employ whatever flavor of rapid application development is fashionable at the moment. This is, I'd guess, the reason behind the cultural gap between embedded/real-time developers and application developers -- the latter tend to think that the former are a bit slow and plodding, not really programmers and so on. The trouble is, if you apply contemporary applications methodology to avionics you get MCAS and the occasional out of control 737. A misbehaving program on a desktop is a nuisance, in real life it costs money and potentially lives.
The general rule in engineering is you get two from three of "fast, correct, cheap". You choose. Personally, I don't mind working on 'last year's model' because I'd rather have something that works properly than something that's packed with features (many I neither want nor need). Website developers please note.
(Sticking people in jail doesn't really make them work any better. Its the system that's the problem, not the people.....)
if you apply contemporary applications methodology to avionics you get MCAS
I suppose it depends on what you call a contemporary methodology - and naming things is one of the great software problems :-)
I feel that one of the big problems that many development approaches have is that they do not have an appropriate verification, validation, and certification aspect to their Software Development Plans.
One reason that High-Assurance approaches can seem rather plodding is that there is someone putting a brake on the adoption of novel approaches by asking - but how are you going to certify that to the necessary standard. If that can't be answered the design needs to be re-worked to use an approach that can be certified.
(That brake does risk becoming a bit of a dead-hand on innovation if people become too dogmatic - but the onus should always be on the system designers to establish how they will certify their system to the required level. If you can't answer such questions the problem is with you/your design, not the person asking the [awkward] questions).
Companies need to also understand that most of their 'security' people should be vetted properlh, not just because they previously held the jobs at other companies. So many of them have little to know understanding of the concepts of security... E.g. either by being over-zealous and stopping people from doing their jobs, or by being weak and not policing the policies that they blindly copied from the internet.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
