Posts by Claptrap314 3215 publicly visible posts • joined 23 Jan 2015
Again, again, and again: get critical infrastructure off the public internet & keep it off.
That is NOT a 100% fix. It IS a way to make it much, much harder to get in where it matters.
Colonial pipeline is particularly obnoxious to me because they ALREADY have a physical network in place. I strongly doubt that it would be particularly difficult to run an IT network though the interior of their pipelines.
Yeah. First off, the initial patch flat did not work in many cases. Second, since we were on Jenkins, I had already implemented firewalls to block access to all but the IP addresses of the sub-handful of users who needed access.
I wrote also our vulnerability & exploit response times to clarify security patches are proposals to be evaluated as one of an array of possible responses. Since we were in a regulated industry, that array included taking everything offline to protect customers.
Hey, El' Reg! You little explainer glosses over the fact that Turkish treatment of the Kurds is what led to the coinage of the term "genocide". This isn't a "sovereignty movement" comparable to Catalonia, or various crazy ideas in the US to carve out new states. These people have seen entire districts depopulated at a matter of official government policy.
Related to Nate Amsden's post, understand that individual DCs at Google weren't designed for HA when there was no GCP, just Google. They found it cheaper to have about 90% uptime per DC & build all of their services with that in mind. It's been close to a decade, but AIR, each DC had a semi-annual two week planned outage. Everything was built to N+2, so the week that we had an emergency maintenance on a second DC that we were in, we got a bunch of alarms, but the customer never saw a thing.
In the two weeks at the end of the year, the we entered a "config slush". No changes were permitted except to deal with an active problem. OMG rate dropped 80%. Every year.
Near the end of my time there, there were two interesting developments. The first was to try to reduce the amount of toil involved in managing a service by having code handling the detailed config changes. The project appeared to take a lot longer that I would have guessed. The second was to try to improved the availability of the individual DCs by adding various forms of redundancy. We SREs were more than a bit skeptical. My PM told me, "It is in Google's best interest that you pretend to believe these numbers."
I get, from standpoint of a general service provider, that very few GCP customers have the chops to actually manage multi-DC redundancy. Therefore, the market is going to demand that individual DC reliability be "good enough". But when you start doing the math, I have a really, REALLY hard time seeing how you can get bet-the-company certainty of three 9's out of a large datacenter. Certainly, it's been almost 30 years since the electrical grid for the Eastern US was down for four hours, but that sort of thing really can happen.
If you aren't hosted both in US East & US West, you're not very much there from a reliability standpoint.
I've always wondered about this. That email is fully under company control. If you make a copy, in any form, and remove it from company control, then you are almost certainly in violation of your employment agreement, and very likely the law.
And it's been more than twenty years since I was at IBM when they implemented a 99-day auto-delete policy expressly to avoid discovery.
I'm not saying that there is no value in documenting email--it can certainly buy you a few more months standing on the plank. But actually getting the documents to court? Sounds much more difficult.
In 1995 or 1996, I decided to use a separate partition for Program Files. I set the appropriate registry entry & proceeded with the install. Guess which was the ONE AND ONLY vendor to ignore the registry entry?
It doesn't matter what kind of "standard" M$ spews out if they themselves refuse to follow it.
I continue to be confused by this. I understand--in general, it is a whole lot easier to rely on DNS even for accesses inside your own domain, but it remains that the DNS record is a second source of truth. Critically, it is a source of truth that relies on a complex infrastructure outside your organization.
Why, oh why, should not primary DNS resolution for systems in your organization for your organization not be an IP address in your organization? It is that important to have SPoFs on the outside?
Let's walk through what was happening at my previous job.
1) We have servers that we want web access to, but not for the entire internet.
2) LE only works if you are open to the internet.
3) ???
What I did when I came in was open the servers up to the internet long enough for LE to do its thing.
We were on Amazon, however, so I figured out how to switch over to their certs.
But suppose we weren't.
---
Yes, in theory cert compromise is a fundamental problem. But in practice, I don't think we've had ten articles about it happening here in the last 20 years.
In the mean time, EVERY one of the bigs pushing this has had more bugs than that every year--most every month.
Try fighting fires that are actually burning.
At my last job, I got handed a significant chunk of our compliance work.
It was bemusing to receive a questionnaire that assumed that backups were not immediately tested.
I don't know what you call an untested bunch of bits supposedly written somewhere without being restore tested, but it ain't "backup".
It is incredibly hard (if not impossible) to remove all single points of failure from a complex system. (And sometimes that SPoF is a human)
Last point first: If your SPoF is a human, you have a substantially broken process. I've seen positions created (by the demand of the board) to fix this.
As for the first, note that it is often much easier to fix this in software than hardware. Ten years ago, at least, Google did not use RAID in their data centers, as the RAID controllers themselves failed too often. Data resilience was handled completely by software.
Likewise, there was not a lot of redundancy within an individual data center. Their policy was, "Outages occur (planned and unplanned). Design your service so that it is not a problem when they both happen at the same time." The week that happened to my service had a lot of noisy pages, but the users never saw a thing.
Efforts to improve data center uptime by adding various redundancies were met with great skepticism, leading to my favorite business quote, "It is in Google's best interest that you pretend to believe these numbers." (Early results were underwhelming.)
Of course the entire reason that DNS can respond with multiple IP addresses for the same service is specifically to remove a class of SPoFs.
----
You had to conduct a risk assessment to determine whether it was safe to have a meeting in a room where the lights weren't working, but had a big window that let light in?
Well...yes. You conduct a risk assessment on every room as it relates to a power outage. That assessment comes back, "The room in question is on the first floor, and therefore accessible without resorting to elevators or stairs. It is also on an outside wall. Therefore, during hours of full sun, use of this room for meetings presents minimal risk." The DR plan can then use this room for business-hour meetings during a power outage without adding safety measures such as flashlights ("torches") or glow tape.
It is precisely this sort of comically-dry detail that goes into proper DR planning.
But why does the PR include such detail? That's standup fodder. I don't fault them for doing proper DR planning. I fault them for putting that kind of detail in the PR. "We are implementing our DR plan" But someone on the board was just so self-important that they had to be shown meeting in this dark room. And then explaining why. <shakes head sadly>
You basic proposal seems pretty good here, but there are some adjustments I would make.
First off, the camera has four settings: No metadata, raw image checksum, originating device id, and rich data.
The raw image checksum produces a quantum-resistant checksum of the image. No providence, however, this checksum can be published as proof of age, and the path of publishing is a weak claim of creation.
Originating device id includes this information in the checksum in a standard format. The device ID proper is held in the hardware, and thus strongly tamper resistant. Publishing this checksum creates a strong claim of creation.
Rich metadata includes timestamp and geolocation. Unfortunately these are fundamentally easier to tamper with. If the device is connected, a signed attestation might be doable, but you would need an entire infrastructure to make it tamper resistant.
"We don't have any access to large critical infrastructure. We don't own transmission. We're a distribution company. Yes, we're part of the overall grid, but the impact of taking out Littleton is small. You would never think that would be a target of any type of attack," Lawler told The Register.
You know, unless you had heard about L0ft's testimony to the US Congress in...1998?
Okay, so it's not the job of the GM to be a cybersecurity expert. But you are designated as critical infrastructure. Somebody in the US government that you support so much thinks you matter. You have to wait for a personal contact that you done f***ed up to believe it?
Okay, so I can appreciate, very much, this part "It sounded like one of those Microsoft scams," Lawler said. He told the agent: "Go f-yourself, I'm not going to click on a link, you must think I'm an idiot. What is your name again?" Personally, as this was from the FBI, I would have gone with, "You ever hear of Frank Abernathy? I'm doing what he told me in this situation." And hung up. Yeah, you call the FBI up after that, since you are the GM of designated national critical infrastructure. Otherwise??? Yeah, that's a seriously unprofessional contact.
No, it is mathematics. Specifically, the claim, "The following code, when compiled, linked and executed on perfect systems, does exactly what it is supposed to do" is a mathematical theorem.
And I'm not talking about the "theorems" we glossed you with in calculus. I mean the real stuff that you only get to in graduate school.
Yeah, I got into the PhD program in Austin. That's about the minimum requirement, by the way.
If you bring people in with the equivalent education of the ones who did the original work, this really shouldn't be that bad. In fact, a lot of old Fortran predates IEEE-754, which means that a lot of algos can be rewritten to take advantage of the guarantees.
Sure, I wouldn't trust this unless someone was a mathematician with a background in floating point, but I cannot be the only one....
AMD 64 had unencrypted microcode updates? Uggh. That was my time, but I didn't know about that. What a mess.
While I can than fully appreciate the desire of hackers to role back the hood and tinker (I actually brought up the idea of exposing the microcode on the K5 to designers), supporting such a feature would just be too expensive.
But what you really, really don't want is an attacker gaining access at that level. Really, really, really.
"To address these issues, the cybersecurity directive mandates that software companies which sell to the government must submit proof to CISA that they are following secure software development practices." So, no Microsoft then.
And nothing based on Linux, or BSD, either. Check those licenses, people.
Yep, this is a bunch of sound & fury, signifying nothing except "We're the ones serious about security, not the Orange Man."
There were literal tons of chemical weapons found at multiple locations. A fact that was not terribly hard to confirm on the web for a few years, notably a major British newspaper condemned the US for exposing troops to it, but every time I point this out, it gets harder to find.
Funny.
For the first 2/3rds of the article, El Reg carries on as this this is a decade-old issue. Only near the bottom do we see that the matter goes back to the 90's. But the stage was being set before that.
In the late seventies and into the eighties, cable was expanding in the US, but the last mile is/was EXPENSIVE. Cable companies sought concessions from local governments to protect their investments, and this resulted in the cable monopolies we have to this day. When the internet came to consumers, it was over the telephone at first.
It was one of those timing things--copyright-destroying (user-to-user sharing--Winamp?) took off just as cable internet was getting started, and was tremendously enabled by it. Big Content was apoplectic, which is understandable, and demanded that these apps be blocked. It did not help that each one of these apps declared that is was so special that it did not need to use exponential back-off--until just the traffic from that on app was enough to congest itself. These new apps were a legal threat, in that the ISPs were enabling the violation of copyright, and a technical threat, in that they were pumping out far, far, more traffic than the switches could handle. The cable companies were already cutting deals regarding content on their networks, so they were ready to do so with the internet they were providing as well.
Which threatened to kill new apps entirely on the Internet.
Up until this point, (late in Bush II era) the US government had taken a very hands-off policy regarding the internet, but this was seen as a crisis. Net neutrality began as a slightly-left-of-center effort to block the ISPs, especially cable companies, from ruining the Internet. When Big Content jumped in, however, we should have realized where this was going.
The fight is almost entirely between Big Content and the ISPs regarding contract clauses. Don't think it is anything else. Breaking up the local cable monopolies would be a good thing for several reasons, but it would mean that Big Content would have way too much power relative to the ISPs. I don't like the implications of trying to redress that imbalance. It may be that the best we can do for consumers is to leave the cable monopolies in place (with more (uggh) regulation) plus NN.
There have been computations. As mentioned elsewhere, it is the job of the various agencies to do the calculations. What you missed in my earlier comment is that if Kessler were at all imminent, the computations demonstrating such would be in the article. They are not, so it is safe to assume that the agencies are doing their job, and that we are not on the edge of such a disaster.
While it is true that a Kessler cascade will happen eventually, it is extremely noteworthy to me that, for instance, the wiki page has 0 information regarding computations of when it is likely to kick in. The simple answer is: "not soon". That said, if we do eventually get there, the cleanup will likewise be some orders of magnitude greater than what is currently envisioned. (For instance: the 550 km orbit is 43500 km long--so the "Musk orbit" is going to be populated with 42000 satellites--that's 1/km)
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
