AWS would prefer to forget March ever happened in its UAE region I received an email / billing notification from AWS this week that may be the most diplomatically crafted communication in the history of cloud computing. Here it is, stripped of the usual boilerplate around it: "AWS is waiving all usage-related charges in the ME-CENTRAL-1 Region for March 2026. This waiver applies …
Yeah, that.can only be a matter of time, before the hypertechs and their bro bajillionares start building those. We've all played those RPGs.
Frankly, I'm sometimes surprised it hasn't happened. Maybe they don't have the cash right now because they're burning it on the AI bonfire? Mercenaries won't accept "stock option" funny money.
I was going to post : Don't go giving Bezos any bright ideas.
Honestly, with his money (and that goes for El Zuck and Ellison as well), financing a personal army would be a question of finding the right general (depending on what you want, you could employ an ex-Marine, an ex-blackhawk or an ex-wagner - there's no lack of candidates) and then signing the check. Problem done.
So no, let's not go tell mega-billionnaires that they can think of building their own personal armies in addition to building their own personal money-printing presses.
Billionnaires have a moral duty to make the world better, they just need to be told.
Politicians are there to make it shittier and they're doing a bang-up job of that these days.
You already have my upvote because I thought the same thing.
My phrasing: I'm sure there are plenty of American good ol' boys (i.e.: gun nuts) that could be Amazon's own private corporate militia if the US feds and various states decided to take up arms against them. If the pay is good enough and/or you can convince them the [ federal | state ]government is the enemy...
But as the first reply said, we shouldn't be giving them ideas, or we might end up like the novel "Jennifer Government."
(Icon: Amazon's newest drones, with guns, to defend the packages and/or truck drivers, or maybe threaten the drivers to work faster.)
The means of destruction aren't overly relevant. We always used the phrase "if lightning struck the system" when discussing redundancy and fault tolerance, but it never really meant lightening literally. Drones and military actions caused the damage, but it's not that different from when a tsunami took out the Seagate and WD factories in Thailand back in 2011. It wasn't just the factories that needed to spin back up, the actual homes of the workers were also destroyed. So the end result looked similar to a war zone. Total destruction of a building (or several buildings in a region) is always possible.
The oil production plants in the region absolutely do have plans to mitigate "part of plant is physically destroyed", because they pay attention to history. Any that bought services from AWS will be surprised and very disappointed that Amazon did not.
In our industry we regularly consider these kinds of event.
Sometimes the result of the exercise is "we only need to handle this well enough to evacuate" because the only plausible causes mean there's no business to recover, of course.
But isn't the solution in the Amazon case that the customer fails over to another AZ or region?
The comparison with the petrochemical industry is irrelevant as you can't just flick a switch and move your processing in an instant to another facility thousands of miles away.
But isn't the solution in the Amazon case that the customer fails over to another AZ or region?
That's what the marketing pitch will have said, yes. And that's what gullible customers believe.
Amusingly, if you look on AWS landing page for benefits they offer, there's a tile for "Reliability", but when you click on it takes you to the "What's new" page that has nothing on reliability. Fair enough.
No disaster recovery runbook in the world has a section titled "Regional Armed Conflict."
Perhaps, but I did read one that had a section on "Civil disturbance", ie protecting the office building from rioting and/or looting.
This was in the DR plan of a California-based sister company, that I was reading in order to get ideas of our own DR plan. As we were based near Weston-Super-Mare , we didn't need the section on earthquakes, or tornados, and their plan for 'Civil Disobedience' wasn't really applicable, as none of our employees had their own personal firearms.
(Yes, the plan for our Californian sister co was that in the event of something like an earthquake, they would ask/require employees to camp out at the office with their own guns to defend the place. They made printers.)
Fortunately they'd never seen Weston when the pubs are kicking out, or they'd have had a whole new perspective on 'civil disturbance'.
As we were based near Weston-Super-Mare , we didn't need the section on earthquakes, or tornados, and their plan for 'Civil Disobedience' wasn't really applicable, as none of our employees had their own personal firearms.
Did it include tsunami and storm surge? Rare though they are in that part of the world there's a bit of history.
Yep, we've wargamed for civil unrest, too. Not so much in the sense that looters are going to run away with our kit and sell it for parts, but more that entire areas could become no-go zones to employees, or potentially worse, that infrastructure could be damaged and workers are unable to get in and restore it. What if it did take days or weeks before an employee, contractor, or vendor was able to access an area because the authorities said "closed for safety."
That's less likely than a natural disaster causing a similar outcome, but the same planning is useful for both.
We're in hurricane alley (US). Quite a few of our employees are licensed to carry personal firearms. The owners have made it clear that's allowed in the office, and we plan for it if the hurricane hits, even though IT has made remote work secure and easy.
Yeah, after a hurricane, we'd prefer people work remotely. But what happens if the hurricane wipes out the fiber in your neighborhood while the office remains functional? What happens if your house is flooded and you'd rather work from the office than whatever awful hotel room you'd be lucky to get when a bunch of housing is wrecked?
Probability of a mega-hurricane in any one year is low. Much more likely everyone just takes an unplanned 48 hours off to deal with one of more likely intensity. But we have discussed it, we have planned for it, as every enterprise should.
Should note that most of our hurricane planning is mundane. We have a backup generator, our entire building has a battery system, key equipment has enterprise-grade UPS backup, and battery packs are given out freely. Our Internet comes in via fiber with backups via cell and satellite. We keep extra water and non-perishable snacks stocked, beyond just what's needed for the breakroom on any normal day.
That covers a lot of other scenarios, too.
We always used the "crashed into by an airplane" scenario in disaster planning.
What we never went as far to consider was the simultaneous loss of two datacentres, which is what has happened to AWS here. That was put into the "give up and go home category" so hats off to AWS for managing as well as they have done.
Complaining about the lack of billing / audit records is a bit churlish tbh. I'm sure auditors will be able to extrapolate from the previous month given the circumstances.
A well known UK retailer used to have its two main data centres situated about three miles apart from each other, neither being that far from Heathrow.
I suppose if both were taken out in one go, the reasons for that event would be so bad that customers not being able to buy a tin of beans would be the least of the problems the country would be facing.
That's exactly why redundancy is important and should be mandated for large and/or vital businesses.
If an event that bad happens, it becomes even more important that customers can buy a tin of beans without being told that the computers are down.
My state (US) has a hurricane problem. Grocery stores and petrol stations are legally required to have backup generators. The same principle should apply to IT. If something that bad happens, the authorities will have enough problems to deal with without also having to manage millions of hungry people waiting for relief because food was on the shelf yet registers were down.
At a US based cellular company in the early 2000s, on of the IT VPs bragged about how much money they saved by putting the DR location in the same city as the primary facility which was next to active railroad tracks as well as the main hazardous cargo bypass for the metropolitan area in a region also prone to ice storms in the winter. I suppose it probably didn't matter though, given the DR plans were all just paper exercises and they were never tested (we all knew if something damaged the primary data center, it would be weeks if not months before they'd be able to get everything working properly again).
"We always used the "crashed into by an airplane" scenario in disaster planning."
A European data center I worked in several years ago was built in the 80s and allegedly was specifically designed with protecting against an airplane crash in mind (clay mounds stacked up to the exterior walls, the building divided into 3 distinct independent physical "zones" (i.e. effectively 3 DCs with separate generators etc).
> Nobody's fault-tree analysis includes "building hit by drone."
In that part of the world, nation-state attacks should have been in even the driest of corporate disaster plans, but even if it weren't a thing, the unlikely stuff is a useful thought exercise in disaster planning. Like you said, the means of destruction are often not relevant, as it's our job to consider whether risks are:
1. Reasonably likely and worth planning for.
2. Probably won't happen, but existing planning would cover it.
3. Extremely unlikely and not cost effective to plan for, or not able.
In that part of the world, drones are a #1 or #2. In Ashburn, they're still a #2, because the "what if" disaster planning should have already covered a terrible accident with natural gas backup generators leaking and exploding, a deranged person with an IED attacking a DC, etc. All those are covered by "how quickly can we transition to another region and is our data safe and accessible if that region fails?"
Not much different than asking what happens if the CTO is abducted by aliens. It's a useful thought exercise in exploring personnel redundancy and making sure business doesn't stop because you lost the only person with a critical password or access key. Of course, the CTO is much more likely to be stabbed to death by a lunatic while getting in his car, but that's not a nice thing to think about in meetings. Thus, aliens. It probably won't happen, but your enterprise needs to be prepared for losing a critical employee.
IT is doing its job when it can sit around discussing risks which probably won't happen and honestly say, "yep, we'd still be good."
I used to work for a large manufacturing firm with facilities spread across the Puget Sound (Seattle) region. A system we had build was based on a set of redundant servers located around the region, with mirrored data. Primarily for load leveling, but if the Renton plant were to blow up, everyone could just switch their connection to the Everett server.
It worked fine until IT management edict came down: All production critical servers were to be installed in a single centrally located data center. Located (literally) a few hundred yards from the Seattle earthquake fault.
> Nobody's fault-tree analysis includes "building hit by drone."
Except maybe the insurers who will have to eventually pay somebody in the aftermath?
I have worked for one of those and they very much do consider how to handle situations such as these.
On the other hand Amazon do everything they can to let their customers believe they are "prepared for this" whilst doing nothing to actually so prepare beyond, apparently, post-hoc pretending that it never happened.
If three 9's and lies instead of transparency are good enough for you then good for you but be aware of what you're paying for.
Exactly. When Corey says
"it's hard to blame anyone this side of the Pentagon for not having that particular item on the risk register"
...he's absolutely missing the point that we go cloud so that the provider analyses for all risks to give us resiliancy. The myth of cloud computing is that if one centre disappears, the other centres will pick up the slack, and there will be no loss of service.
An asteroid and an earthquake in a previously geologically stable area are 100% unpredictable, but we still want to know there are disaster plans that cover them, but Amazon have failed to account for a very plausible threat, and yes, a company that puts massive amounts of service in one place absolutely has to be aware of the fact that they have just created a very viable target for anyone attempting to sirupt commerce.
But here's where the article truly went into dystopian territory:
"And honestly? It shouldn't. Try putting that into your company data recovery plan, and you'll get uninvited from business continuity planning faster than I was at investment firm BlackRock when I pointed out that if both San Francisco and us-east-1 were unavailable, absolutely nobody was going to care about our roboadvisor for the foreseeable future, if ever. (I can't imagine why I was let go from that job.)"
So he got let go because he was talking about genuine risks, and he thinks that was right??? He's saying that because management want to stick their heads in the sand and ignore genuine, realistic risks, then companies should let them?
The whole problem with the capitalist system is that it encourages ignoring risks and cost-cutting, and if people are going to defend AWS systems in three datacentres falling over when one datacentre experiences a total failure because apparently nobody could predict that a massive part of many business's infrastructure was at risk of being hit in a conflict-ridden part of the world... well that's just letting them off the hook. We all need AWS to fulfil its promise, and we are trading the increased risk of being in a shared facility with the decreased risk that Amazon's infrastructure claims to provide,
It has failed.
The entire premise of cloud infrastructure is that you're renting capacity from someone who handles the physical concerns so you can focus on your software. That contract didn't account for the data center becoming a military target, and .
The status page cannot be parodied, it mentions everything but the cause. It's as if it described a car crash as a rapid deceleration and a sudden reduction in length of the front crumple zone.
Apart from:
Mar 02 4:22 PM PST We are providing an update on the ongoing service disruptions affecting the AWS Middle East (UAE) Region (ME-CENTRAL-1) and the AWS Middle East (Bahrain) Region (ME-SOUTH-1). Due to the ongoing conflict in the Middle East, both affected regions have experienced physical impacts to infrastructure as a result of drone strikes. In the UAE, two of our facilities were directly struck, while in Bahrain, a drone strike in close proximity to one of our facilities caused physical impacts to our infrastructure. These strikes have caused structural damage, disrupted power delivery to our infrastructure, and in some cases required fire suppression activities that resulted in additional water damage. We are working closely with local authorities and prioritizing the safety of our personnel throughout our recovery efforts.
A war that takes out your site is no different than an earthquake or hurricane or any other disaster. Regardless of the cause, your site is gone. The span across the Middle East in which Iran is operating is no bigger than what a decent hurricane could cover, and they make DR plans for such events. There is no excuse, therefore, for them to not plan for a war hitting multiple sites.
More to the point, probably 90 percent of warlike aggression since 1980 has taken place in the Middle East, making it doubly bad that there was no DR plan for a few missiles lobbed into several data centers on the same day.
The point not yet addressed - an entire month orlf cloud computing evaporates and what was the real effect? Unless the story just didn't go into it, it sounds like a monthlong non-event.
Never mind the threat of conflict to civilian DCs, your army secretary Dan Driscoll is publicly quoted on the idea that in addition to building military AI data centres regionally including outside the US, he's stated "in a contested environment, we can actually go build a lot of these things in the theatres where our soldiers are”.
So I'm guessing modular AI DCs, and your politicians think they can transport these to wherever they're at war with brown people. Hardening, point defence, cooling and power supplies should keep a few people busy, and I'm sure nothing can possibly go wrong.
https://www.ft.com/content/332c1134-18c4-4e80-a9c3-06aa0c20513e?syn-25a6b1a6=1
Link should work - FT is paywalled, but unless you've exhausted the handful of free articles per month (or nuked cookies) you should have access to the article.
> More to the point, probably 90 percent of warlike aggression since 1980 has taken place in the Middle East, making it doubly bad that there was no DR plan for a few missiles lobbed into several data centers on the same day.
I would suggest that if you realistically believe your DCs will be subject to warfare then the appropriate solution is not to have DCs in that region at all, not we'll build more DCs and put more staff in harm's way.
'Nobody's fault-tree analysis includes "building hit by military drone." No disaster recovery runbook in the world has a section titled "Regional Armed Conflict." '
Err. Ours does. And I can assure you, we are not alone in this. (I could tell you more, but then I'd have to kill you, etc...)
Working with a company that was considered part of the Critical National Infrastructure, they had to have comprehensive BC/DR plans. They had a London site that worked at strategic level (overall site status monitoring, back-end systems, longer-than-a-day planning, etc), IT in London and in a remote datacentre, and the big industrial site which handled tactical level (per machine monitoring and management, on-the-spot responses, etc). The continuity plan to keep the industrial site running in stand-alone mode was written on the basis that all London IT is destroyed and all London staff are dead.
I once worked at a large organization that stored data backups in a salt mine. One of the stated reasons is that (disused) salt mines are resistant to strategic nuclear devices. (I don't know if they ever wrote that into a document, but I wouldn't be surprised.) So mayb "Regional Armed Conflict" can be subsumed under "Global Armed Conflict", which will be next month.
And did your company have a plan for how, after the nuclear conflict, they were going to provide access to the backups, identify authorized people when their identifying documents had been nuked and you couldn't call to authorize them because there was no phones, provide facilities for the skeleton staff that was doing all this because the primary staff had been killed in their homes, etc? There's far more to a disaster recovery book than "maybe people want our services because of disasters".
Many of the runbooks I've seen tend to gloss over parts like this, which in some ways makes sense because in a war, how they operate is going to depend on exactly what they have left and it's difficult to predict that. Stuff they never thought would be destroyed will have been whereas luck will have preserved several important things. That's not ideal, but things rarely are when people are intentionally trying to make them as less ideal as possible. Preparation is always useful, but a lot of people pretend they've done more than they actually have or think that writing down a plan is all they need to do.
A long time ago I tried to persuade our fledgling DR committee to define a "worst credible incident". Anything beyond this would be by definition INcredible and so we would not need to worry about it. I was surprised by the amount of pushback this received; "we can always do something". Not if we're all dead.
> define a "worst credible incident".
Probably a meteor strike as "probable" and "major" at the same time. Or a Solar Flare taking out satellites and power grids.
It would also have global consequences meaning that overseas aid won't be coming any time soon as they will be in the same boat.
"Not if we're all dead."
Some dialogue from a sci-fi comic strip once:
Smart guy (engineer-type with military-like experience): What's the defense plan if you lose your eyes [sensors] here in the [star] system?
AI in charge: [Detailed reply about defenses.]
Smart guy: "I've defended my eyes" is not the same as a plan to make do without eyes.
AI: You might as well be asking "What's your plan for victory if you get defeated?"
Smart guy: That's a fair question. Do you have an answer?
In my work as a security architect I occasionally get asked by an assurer or auditor why I think running AWS infrastructure in just two availability zones without a second region is enough. The latest was just earlier this week. It shows that they do not understand risk/impact balance outside their own little box. I have to point out that if something can take out two geographically separated data centres simultaneously then the impact is not restricted just to their website, and they probably have bigger problems to worry about. Some of them accept this. Some still think another region would help.
I worked for a small public sector body. An auditor once asked what would happen if both our main and DR sites went dark. I said if that happened, something very big & bad was happening and no-one was going to care about our organisation.
Auditor ticked their box as we had clearly considered the possibility and we had a plan. (Do nothing is still a plan!)
In normal circumstances AWS warrants to destroy and data on media that are no longer used in order to satisfy requirements for customers of highly regulated industries. In the event that the datacentre is out of service due to a drone strike, I'd assume that there are probably a sizeable number of media devices on there that still have potential custome data on. I know encryption at reast should still help mitigate the risk, but it poses an interesting legal challenge.
I switched to this article immediately after reading a recent column by journalist Dave Barry. The continuity of style was remarkable.
Arguable relevance to anything at all: in his current column Dave describes going ONLINE to consult a witch. As it happens, he was consulting a UK-based witch about a US college basketball game, so the purchased spell was unsuccessful. But it was all ONLINE.
A journalist did a story on Esty 'witches' who do online consults, and bought several curses aimed at Charlie Kirk. The story was published two days before he was shot. So maybe they are effective?
> Nobody's fault-tree analysis includes "building hit by military drone."
That depends on your line of business. The Pentagon will surely recognize that risk, having had a plane flown into it once, and the resilience being shown by Iranian C&C at the moment suggests that it occurred to them, too.
The company I worked for when 9/11 occurred that was headquartered next door to WTC. The building was severely damaged but the business kept running. The DR plan doesn't need to say what hits the building just that it has been hit and hence no longer available.
I'd gone on a business trip a couple of days before and my wife and kids had gone back to the UK. I was asked to stay away and ended up working in London for the next 3 months. The door man on our apartment in NYC looked like he'd seen a ghost when we finally returned as they were convinced we were amongst the missing.
(I can't imagine why I was let go from that job.)
You were ejected from the committee and fired due to knee-jerk, thoughtless, primitive, monkey-brained emotionalism. "No say bad thing, scary, make bad thing happen, go away!" a.k.a., "shooting the messenger".
It was your duty to raise the points which you did.
It was management's duty to consider those points logically. They failed.
That's open to dispute given Bezos' closeness to the man who *is* responsible for this situation- Donald Trump- and the fact he's actively supported and enabled him via donations and backdoor bribes (e.g. paying them tens of millions for the 'Melania' film they knew damn well no-one would watch, but made for reasons of favour and bribery regardless).
Bezos enabled Trump knowing damn well he was pathologically unsuited to a position of such responsibility, so he can certainly be held responsible for the consequences.
But Trump and his ilk didn't foresee that Iran could attack all around, instead of trying to attack better defended positions.
They are probably the same analysts who wrote "Afghan troops will stand six-nine months before being defeated by the Talibans", "Putin ammassed 200.000 troops but won't attack Ukraine", and "Ukraine could stand a few days only".
Trump sacks anyone who tells him what he doesn't want to hear and surrounds himself with 'yes' men.
It's also quite possible- if not probable- that he started the war purely to distract from the Epstein files.
He just didn't care enough to think about it in anything other than disinterested, self-serving and short-termist terms.
This is on Bezos. But it's also on the large majority of Americans (who either voted for someone manifestly unfit to be president or who didn't care enough to vote against him) and on the United States itself.
It was widely believed that the Israel military used those AWS data centres, and given the belligerent history of Israel in the region (& that it has been obvious over the preceding months Israel were gagging for war with Iran) then it would have been obvious to all but the most dim / credulously optimistic that that those data centres could well be targets due to their use by Israel.
People forget, it was only twenty years ago! At least one company had production and DR separated by a mile or so, which would normally be acceptable, if only the large oil refinery right next door didn't decide to explode.
For some risk plans, the response to e.g. your main head office being hit by a plane on a day everyone significant is there may be 'go out of business'. However, a datacentre being wiped out should be able to be coped with.
Work have more than one datacentre, they're a moderate distance apart and separated by hills; in the risk profile it's very unlikely both will be taken out - but not impossible. If for instance they were both at opposite ends of Birmingham, and Birmingham (and only Birmingham) was somehow nuked, whilst there would be a lot of other things to worry about the sad truth is outside Birmingham life would go on.
You'd better hope you can restore from some offsite (different country) backup at the very least, because the alternative is hundreds of companies being affected, and hundreds of staff including yourself being out of work. The government, on the whole, aren't going to be nice about your work being dependent on a nuked city and have other things to worry about. Insurance is unlikely to cover it either.
"No mention of the Iranian drone strikes that physically destroyed two of three availability zones in the region on March 1st."
Um, Amazon announced that the DCs had been hit by drones on March 2:
https://health.aws.amazon.com/health/status?tag=pcmaguk-21&ascsubtag=u%7Ccloud-infrastructure%7C163488%7Camazon-cloud-services-disrupted-in-uae-after-objects-hit-a-data-center
There are posts which claim multiple datacenters being destroyed could only happen predicated by something far worse. This is true, unless the datacenters themselves are the target.
In the present case there is evidence the AWS datacenters were the subject of a precision attack using missiles and drones. Namely, loss of the datacenters was not collateral damage from a disaster that destroyed the rest of the city.
Iran has claimed it will be economically so costly for its enemies that those enemies will admit defeat. The idea is citizens of a secular nation place all their trust in money and will do anything to avoid a lower standard of living.
From this point of view if blowing up datacenters with targeted munitions (not too mention shipping in the Strait of Hormuz) stops commerce, then this will be enough to obtain significant military concessions and win a war.
Given the lack of international response to acts if war which focus on civilian targets instead of military ones (see also the war in Ukraine) and the asymmetric advantage, I would expect blowing up datacenters to become a more common tactic.
Eh? No one uses the billing an invoice as a canonical infrastructure manifest. Not only that, but it doesn’t go low-level enough to, and usually it’s a different department handling billing than Engineering - so that’s nonsense.
I rather suspect that their usage data is so (understandably) dirty as to be indefensibly unusable so they CAN’T share it with customers because it’s totally unreliable.
Infrastructure can be abruptly and permanently wiped out by all sorts, and planning should always assume the worst.
It's not like it's even that unusual for a plan involving a DC to have to consider sonething like 'indirect fire' or other attacks and it's an entirely credible threat in some cases.
Certainly plenty will have considered that someone might be keen to destroy the things.
Weird then that AWS just didn't imagine the possibility of a large, catastrophic permanent failure of a couple of sites, it's not like it never happens.
This post has been deleted by its author
"No disaster recovery runbook in the world has a section titled "Regional Armed Conflict.""
During the 30-or-so years of "The Troubles" in Northern Ireland I would assume that the DR runbooks for various government agencies (Civil Service, Police, Army) as well as some commercial organisations (i.e. BT for their exchanges and network) *would* have considered plans for a car/truck bomb damaging or destroying their infra etc
I'm not sure if the reason that BT built their telephone exchange on the Lisburn Road in Belfast underground (late 80s?) related to such concerns.
During the 30-or-so years of "The Troubles" in Northern Ireland I would assume that the DR runbooks for various government agencies (Civil Service, Police, Army) as well as some commercial organisations (i.e. BT for their exchanges and network) *would* have considered plans for a car/truck bomb damaging or destroying their infra etc
BT has always had that. Which lead to the occasional excitement, like an ND when an armed guard dropped their rifle in an exchange canteen. Or having a large datacentre in Harmondsworth, just outside LHR. So when I was at BT, there were I think 5 large DCs, pretty widely geographically seperated and also Sunguard that ran a rent-a-mainframe and space for DR. Then regular DR exercises to shift workloads (and workers) to an alternate site and get things up and running again. Which also included some fun what-ifs. Like being told I could sit out one test on account of being dead. Yey!
But also why I'm glad I cut my teeth at BT given they've always been CNI, so done DR properly. Didn't matter what the reason, this site is gone, now route around the problem. Then leaving and discovering businesses that may pay lip service to DR, or regard it as an unneccessay cost. We've never needed it, so why spend money on it? So I'm not at all sympathetic to AWS, especially as they market cloudybollocks as making all that stuff easy (but not cheap). Then a risk that's sometimes overlooked, like what happens if the datacentre hosting your business also has deals with the DoD, or Mossad? Which then makes the datacentre a legitimate target. Same thing can happen if you do deals with customers that might have a bad reputation, or even a good one and then attracts a lot of hackers, or DDoS events that have to be managed.
"discovering businesses that may pay lip service to DR, or regard it as an unneccessay cost."
Yupe. I worked at a European-wide telco in a subsiduary which provided managed "hosting" services (they had 3 DCs across the EU) to each of the national divisions of that telco. The whole time I was there (multiple years) none of the country-specific projects I worked on, or was aware of, ever deployed in more than a single DC (as obviously it was cheaper to deploy to just 1 DC).
In this world, somewhat absracted from the coalface of war, one where drone driven domestic terrorism looms large, datacenters being hit by drone strike should be on every country critical infrastructure mitigation plan.... to state not, does a future injustice we have a funny way of stating the very human notion "that'll never happen here" . Then it does.
