IBM, i shoulda known.
there's a reason they're prohibited in QLD, you know.
oh well guess el reg saw it coming and had this ready to go.
Australia is today conducting a contentious national census, and things aren't going smoothly. The nation's Bureau of Statistics (ABS) has hired IBM to run the online data collection portal and at the time of writing The Register's antipodean outpost, and plenty of others, are finding it impossible to reach the site. Your …
If El Reg couldn't see this coming then I would be changing news outlets. Blind Freddy could see that provisioning for a million people per hour isn't enough when most families will get home from work, eat dinner, kids in bed then log in. The saddest part is that because they make names compulsory, the results will be less than honest, negatively impacting public policy decisions for the next 5 years.
The actual deadline for completing the census is 23rd September, so anyone who fails to do it today will not get fined and can complete it in the next 4 weeks or so.
Unfortunately the messaging from the AUS Gov seems to have forgotten to point out that actual deadline day.
So this is essentially a self inflicted Denial of Service attack.
What are the penalties in Australian law for this, and who in the Australian government census "marketing" department is going to get prosecuted for this?
And common experience (see the UK historical census data being put online for public consumption - all over 100 years old of course) as a classic example of under provisioning and failing to anticipate demand). The UK tax return is similar - massive spikes in the few days before deadline(and failed service) - but if the deadline is there, then surely it should be honoured - none of the documentation mentions that people who fail to log their returns will be excused if though demand the service can not be provided.
Now I agree that anyone with an ounce of sense will have filed well before the deadline - little is likely to have changed in the last couple of months, but a little sensible social engineering combined with proper provisioning (hey - isn't that what the cloud is meant to provide?) right alleviate things.
I realise you are not directly addressing the census but, for full clarity, you cannot complete the census in advance - it is intended as a snapshot of the nation on a particular day. As such the demand for the site should have been anticipated well in advance. Or, if it is a DDoS - as they now claim - then that possibility should've been anticipated as well and had suitable defences organised.
As with many others, I cannot connect. There is a phone number you can call.
The recorded message says (a) there’s too much demand, try again after the 10th; (b) you won’t be fined for competing the census after Census night.
Tried again later. Now the recorded message says that business hours finish at 10:00 pm. Try again tomorrow or visit the web site.
I’ll bet nobody saw that coming. Oh wait. Everybody saw that coming except the ABS.
You misunderstand - perhaps deliberately.
The problem that is being highlighted via use of this hashtag - and the numerous other ways the failures have been reported and commented on - is that the ABS have taken the arrogant stance of: "trust us - we are unimpeachable and your refusal to trust us will be punished" and then showing themselves to be utterly unworthy of that trust.
They defended their unprecedented decision to retain personally-identifiable information by telling us that they had never had a breach in the past and then, when their entire submission system goes offline, they cry: "it's not our fault - we were hacked!".
The claim that the ABS's defense of previous hacking attempts should somehow be taken as a guarantee of their ability to resist future hacks deliberately avoids acknowledging one very important point: that the data they are collecting this year is - by their own admission - far more 'valuable'.
However you slice it, they have proven that they really cannot be trusted.
If the systems were subject to a genuine, malicious, DDOS attack (and there is no independent proof of that, yet) then they have shown that they either underestimated the size of the target (and were therefore under-prepared and under-protected) or they did not do the due diligence to ensure the system being built to capture our private information was fit for purpose.
If the systems weren't 'hacked' then it went down either due to the traffic exceeding the ABS's expectations, which again means they do not understand the data they are dealing with or, again, they failed to do due diligence to ensure the 3rd-party solution our tax dollars paid for was up to the task.
With the retention of names and addresses, this census will create one of - if not the - largest single collections of sensitive, personal data in Australian history. This raises questions and concerns and the answer from the ABS has been a revolving door of silence, arrogance, condescension and bravado.
If, as they demand, we view them as competent due to their past competence then I find it logical, fair and prudent to view them as incompetent when faced with such overwhelming evidence to their current incompetence. After all, if your proof of trustworthiness is based on being infallible then why should we trust them again after such a colossal f$#k-up?
Some may argue that it's unfair to judge them by one mistake but when you're dealing with the sensitive information of the entire country - information of inestimable value to eager spies and malicious actors - you don't get second chances. You can't wave it away and say: "we'll do better next time". These eggs cannot be unscrambled.
The collection, handling and storage of this data must be perfect at every stage and for the entire life of the data. No mistakes, no oversights, no cut-corners, no assumptions, no miscalculations, no loop-holes, no bugs, no cracks, no absent-minded omissions.
If it fails anywhere, even once, then the damage to could be immense. You don't get a second-chance at a perfect record.
"Thank you for participating in the Census. The system is very busy at the moment. Please wait for 15 minutes before trying again. Your patience and cooperation are appreciated. [code 9]"
Not that I want to fill it in online anyway given the other articles, but also, filling in the paper version sounds like it'll be added to the same insecure dataset anyway (and all manner of things done to it too).
Contrast that with the UK's 2011 Census ( pop'n about 2.5 x that of Oz) :
Approx cost of some £495m
Thousands (tens of?) of returned and completed forms lost before census day.
Thousands (hundreds of?) of useless forms, (duplicate or non valid addresses) issued in advance.
Return rates in a lot of London Boroughs down in the 60%'s.
Successful prosecutions for non-return? Very few if any, i guess.
Yep, the Aussies have some way to go to match that "success" rate.
This is a great example of how IBM's strategy to move all the people who actually deliver their solution to India is a recipe for disaster. The Indians would have done the testing but with no knowledge (or indeed interest in getting that knowledge) of the local requirements. Ask any existing Australian customer of IBM and you will get a story of declining services and the removal of key people at the whim of the US based accountants.
IBM's role in the exercise is also likely to be questioned - it was paid at least AU$9.6m to design and implement the eCensus solution.
<sarcasm>This has to be a great ad for IBM's cloud services... Not able to scale quickly. Not able to handle DDOS. </sarcasm>
I thought these were some of the primary reasons cited by the Cloud sales people as the benefit of Cloud services over in-house.
There was no DDOS
They budgeted for 1 million 'form posts' per hour - was this users or HTTP POST requests?
It would not be unexpected to have 1 million people completing the census at the exact same time, 6 million per hour is a more realistic estimate.
As for the claims of a 'foreign hackers' - geo-blocking at network gateways is pretty common. By definition only people at home in Australia on the night should have completed the census, so no one outside Australia needed access to the site. And if you don't know your friendly Network Engineer, even CDN's normally offer geo-blocking.
Apparently they could only handle 1,000,000 form submissions per hour, and it doesn't take a statistical genius to work out that more like, 10,000,000 people would log on after dinner, which would be probably between say 6:30pm (EST) and, well I guess most folk had given up by 7:30 pm (I am guessing word was out by then and SA, NT and WA didn't bother).
- I mean some people have real jobs and can't log in during the day like our PM evidently did.....
"Asked about the ability of the online census database to cope with such high traffic numbers, an ABS spokesman said online could handle "1,000,000 form submissions every hour. That's twice the capacity we expect to need.""
So the ABS is claiming some mystery DDOS attack, yet there seems to be no evidence of that at all
If you look www.digitalattackmap.com for last night there was no recorded DDOS attacks in this country.
Also if you are paying $10M for a cloud service
a) I would hope the SLA the ABS should have with IBM would mean they can get a fair amount of compo
b) That someone would have thought about a DDOS and designed the system to cope with is.
Update: I see in the press conference that they had, they are now blaming an overloaded switch.
so they obviously have never heard of redundancy, or distributed infrastructure
eVoting will work like a charm. It will cost much less. See since the votes will be counted way before the actual elections happens. This is to make sure that the right people vote, this the web site does not actually need to record any thing. It just needs to be a bland two page web site.
@AC said "The actual deadline for completing the census is 23rd September, so anyone who fails to do it today will not get fined and can complete it in the next 4 weeks or so."
Which is true. But not useful. The whole point of a census is that it provides an accurate baseline for other statistics. That means the "hours worked last week" figure really needs to be completed on or shortly after 9 Aug, when the hours are clear in the respondent's mind. Not on 23 Sep when the respondent's recall will be inaccurate.
The original intent of six weeks was to allow for postal and other delays for a small group of respondents, not for a significant proportion of the population.
Back in October of 2015, the ObamaCare web site was down pretty much most of the time because they didn't do an incremental roll-out, similar to your census problem.
Even after getting logged into the OC web site, the usability sucked.
You'd think a web site that cost $2 billion would be damn near perfect. Ha. You'd be wrong.
Biting the hand that feeds IT © 1998–2021