Natwest net and phone banking goes titsup Brief Natwest's online, mobile and telephone banking services are having an unscheduled rest today, Saturday, November 5. At time of writing customers who try to log-in to their accounts are greeted with the following message: We're sorry our Digital, Mobile and Telephone Banking services are currently unavailable. We're …
Yup, I work with several finance companies, and all of them are shedding IT staff left right and centre. Unsurprising that they've got the people who cost the less, who are overworked developing stuff at the moment - and then we get fall over after fallover.
You can expect Lloyds \ HBOS to be next, if it's a similar deployed package that many of the banks that have been working on to cause things to collapse soon.
This problem did not commence at 7 am as you stated, scheduled maintenance in the form of a large software update commenced at 01:30 this morning and at circa later the message reminding their customers of this was replaced by the current message. I have been informed by Natwest that all accounts are frozen as they were yesterday and that the problem is affecting their systems globally.
This suggests that the problem lies deeper than the web Customer Self Management and Customer Self Management and Customer Relationship Management systems that the automated and advisor assisted telephone banking services use respectively.
Testing shall commence at 12:00 and while unable to confirm a time for the services to be restored their is a rough estimate "of some time this afternoon".
For those of you like myself who through a quirk get paid today with direct debits also being taken out over the weekend it shall be interesting to see how this one pans out.
Source: My insomnia.
Read the article - there is nothing to correct:
"According to Reg readers, the service was unavailable at 7am."
It merely states that there are reports the service was not available at 7am. It doesn't state the outage started at 7am.
Later, some people might report they found it unavailable even earlier than 7am (as seems to have happened in the comments section). This does not make the earlier statement incorrect.
Also, there is a difference between schedules maintenance - which according to several sources, was supposed to have ended sometime in the early morning - and what seems to have turned into an unscheduled event - to end later in the day (or even later than that).
Just because the initial outage was scheduled doesn't make the whole of it so. As soon as it outruns it's designated time slot, it automatically becomes unscheduled downtime.
A technical argument for managing customer expectation is always due to fail particularly in banking, where people are actually unable to travel home because there debit & credit cards are not getting authorised.
While I value you technical opinion the gross majority of RBS/Natwest's customers see the customer experience, not an ITIL breakdown of pain points in an SLA.
This post has been deleted by its author
Suspects:
Foreign government, motive, oh hai, just noticed your door was open and was curious.
Al Quada, motive, plunge the infidels into chaos.
Anonymous, motive, vengeance for wiki leaks / 4tehlolz
Organised crime, motive, cash money.
Bond super villain, motive, world domination.
Government, motive, big brother upgrades.
Software developer, motive, none, unintentional, banks systems unable to cope with ligitimate user requests from new smartphone app.
Banks offline, SSL cert authorities pawned, RSA encryption seeds stolen, Military defence contractors burgled, nuclear reactors sabotaged, something new and exciting every day.
Script kiddies, organised crime and governmental hacks all blurred together, exploiting networks and systems that were designed for utility rather than security.
This will end badly.
The Royal Bank of Scotland is also down at the moment with the same message. Natwest is now just a part of RBS so it makes sense that they are all down together.
Frustrating stuff, but as long as my money is still there after I am ok about it. just a few years ago we didn't even have ibanking etc. so whatever.
Is probably in place, I know where I work we have rolling business continuity testing. However getting it to fallover seamlessly in an actual emergency situation is another story altogether. Often it works fine others it can take a few hours. You can't predict every part that will fail, just have to practice with the scenarios with think might and deal with others on the day.
There is probably no redundancy per-se at this point in time - the systems being upgraded will be separated from the DR site, with the DR site as failback, there will also be snapshots of disks at the production site. Having a failback site doesn't mean that it'll be quick though, it's highly likely that many systems are having changes applied to them, so failing back manually is a complex procedure.
Having been present on several recovery management calls at RBS, I'd say that they will almost certainly be backing out. Fixing forward is not something that is done at RBS, unless there is no alternative. If they go off SOE (Schedule of Events) the decision is more often than not to follow the backout process, if at all possible, as fixing forward brings unknowns into a meticulously planned change.
Anyway, as mentioned, I previously worked for RBS, until my job went to India. We were having problems with the quality of staff in India then and from what I've heard from people still there, it's not got any better. I wonder if this is the new management reaping their just rewards?
A lot of stuff going on at the same time - BBC main page, HSBC, NatWest/HSBC.
Good for conspiracy theories.
My conspiracy theory is that all the banks are scheduling their upgrades to take place at the same time so if it all goes nipples to the sky no one bank stands out as incompetent.
On the other hand this may be a global conspiracy which will make the occupation of St. Pauls seem like a minor event.
Or not.
The fault is online banking, telephone banking, cash machines and paying for goods via debit card and credit cards. I suppose to pay my rent today and haven't been able to get to my bank account since 1.30am. I can't even buy any food at my supermarket and might have to borrow some money of my mother in law to get some food to feed my family.
The person who is in charge of this so called technical department should be sacked and also who authorised this update on a Saturday morning should be sacked. I thought updates like should have been done on a Sunday morning so minimise problems like this.
Your rent is due on a non-banking day?
Have you called RBS and asked if they can authorise an emergency payment - they can even do them out of ATMs with no card, so you don't even need to go to a branch while it's open.
When would you rather they do essential upgrades? During the week? Maybe they need to carry out two updates to systems over the weekend, it's not unheard of. Possibly they needed to reserve Sunday for contingency, in case of any problems? There are also constraints with backups and the amount of time they take to run. RBS have world class uptime and their changes very rarely go wrong in such a way which would be visible to the customer. They don't do changes without meticulous planning, so I'd imagine that a Friday/Saturday change was essential in the larger scheme of things.
After the HSBC outage, I was just thinking how glad I was it wasn't us (I work at RBS). And now, it is...
In case anyone thinks it, I don't know what happened here, but the mention of "scheduled maintenance" overnight does make me wonder if it's related. I really hope not, as any big failed change usually makes our lives hell as they slap some extra change control on everything for a while.
i am a customer with NatWest and I am due to be paid today i am taking my children on holiday today buy until they fix this I am unable to go anywhere my children one of whom has learning difficulties will be left disappointed if they do not sort this out what where they thinking doing any changes to the system on a day like bonfire night when people need their money all money hits NatWest accounts at 3.30 am so why didn't they carry out the maintenance after 4am .
Three times is enemy action.
You know I have to wonder about the decisions made by the UK banks in regard to security. My bank, HSBC, has just changed its security system to use a small PIN generator device from China. Now its not too hard to put in a few little extra bits in order to enable some sort of 'nullify and reset' signal, or even, given that Chinese companies (and government) have the algorithms to hand, a brute force type of attack. Given also that from my conversations and letters to him, the security head of HSBC, is either a Oscar worthy actor or as dumb as a stump, whats the odds that they've checked the devices properly?
From my point of view; given the chance to bet £10 on a bank paying out money to have the things taken apart and checked from top to tail, or Commander Sir Samuel Vimes Vs a Quasi-Demonic entity of extreme power; I have to say I would be voting for the guy in the chain mail...
Now, how long do you think it would take for our nice tea drinking, cricket loving civilization to turn into a cutscene from Halo:Reach if all the banks that used a similar system that was obtained from China (and whats the betting thats alot of them) were all hit at once. People cant get money for food, bills or petrol/Diesel - They cant get paid, and so on. I know there are other systems in place - but the thing there is that the banks have automated so much, that what was the entirety of a bank 20 years ago is a shadow of its former self (the physical staff etc) and it will *not* be able to cope for an extended period alone. Thats not even to mention the additional havoc that could be caused by pulling a stunt like that in the middle of Great Depression II...
I know it sounds paranoid, but the thing about being paranoid is its like being a pessimist. You are never ever disappointed.
if the Chinese could pull something like that off for an extended period then you wouldnt even need a nasty war like was going on in Libya. You just wait a little while and then saunter down and pick up the pieces after the countries you want are practically in the middle of their own little civil wars.
Even the most elementary tactics tells one that you don't build an army with only one type of unit, you just don't do it. Similarly, you don't put all your eggs in one basket when it comes to bank security - and I wonder how many people, when they look at the their security pin device see 'Made in China' on the back. There have already been cyberattacks traced to China mentioned in the media, given that we probably only hear about 1/10th of the total that happen, Im not happy about the odds that the 'system failures' we hear about are as innocent as they seem...
Time to find me an old sock to put under the mattress methinks.
Sometimes I wonder if board of directors of these financial sector organisations are, well, up to it and IT?
Imagine if the Robin Hood tax went ahead.
All that would happen is that one ruthlessly efficient organisation capable of redistributing wealth from the poor to the wealthy (banks & finance sector) would have to pay up to another ruthlessly efficient organisation capable of redistributing wealth from the poor to the wealthy (hmrc, whitehall & UK tax structure).
And as both organisations seem to place incredible priorities and importance on remuneration of employees and preferential employment practices (with bonus) I cannot see how the public can improve their lot.
Besides there is only a limited sum of capital on the go at any one time.
Banks do not produce wealth, merely take some dosh from some people and pay to other people (after some -ahem- modest redistribution of course).
All that is likely to happen is the finance sector will be collecting taxes for and on behalf of hmrc; from the very public that insists Robin Hood tax is imposed?
History shows that the banks will sooner bankrupt nations than forego employee reward structures. A matter for the bored board methinks.
A show of hands for the call centre staff who have been fielding irate customers all day, with Natwest management not saying anything apart from the ambiguous statement on the site disussed in this thread.
An update is due at 18:00. I wonder if there is a connection with the 3D secure compromise they had a week or so ago. Natwest meets Sony, now that would be fun.
Saw this when logged in:
"The service will be undergoing essential maintenance on Sunday 6th November between 3:45am - 7:00am. Customers may experience some disruption. We apologise for any inconvenience this may cause. "
What are the banks up to I wonder....
yes but does it work paying via debit card as my current balance is 0.00 as off yesterday and normally! if this was updated i would have had a regular Saturday payment today! will this cause any further embarrassment trying to pay for something, and getting refused because nat west haven't updated the balance
my funs were due to clear today but my dd card was decalined at asda rang them up and was told no funds had cleared today but should clear when system is working in a few hours this was at 8 this morning was told 10 them 11 now sometime this afternoon i have bills that go out today and need money
It's strange that absolutely nobody mentions cash. Or having spare funds. You know, old fashion type stuff. Where you take care of your own hide, look ahead a little and don't rely to the extreme on what are, very convenient systems - I must admit.
Doesn't anybody keep some emergency cash in their house anymore? Banks are made of real stuff - you know, humans, bricks, machines. All that stuff which will fail from time to time - even if temporarily. A little bit of preparation, a bit of spare cash stored would not go amiss in situations like this. And yes, I've been burgled several times - and I still keep few emergency banknotes in the house.
Also, so many people mentioning 0.00 bank balances and waiting for their Saturday payment. Where is that buffer amount of money, just sitting in the account - just in case? Is everybody spending like there is no tomorrow - or relying on everything working perfectly every time - with no unforeseen circumstances? A bit of adjustment in life style maybe?
Civilisation has brought us a lot of very convenient stuff - but seems to be turning some humans into real cry babies - unable to take care of themselves and be prepared for a bit of reality.
I'll get my coat now. Sunday preaching over.
are unemployed, disabled or unwell. They may or may not be scrounging on benefits according to your position, but the reality is that many people have £0.00 balances (or less) and the timely payment makes the difference between their homes being warm or cold tonight.
I assume you have a rainy day fund by your post, but when nearly a quarter of people in this country are in fuel poverty (technically 10% of their income on heating) but with reality that the quantitative values on fuel poverty are that the top 20% of them are spending 50% on heating the cliched "Heat or Eat" comes to mind.
No doubt you are successful within your domain and warm this weekend but if like 82% of the UK workforce you have a one month notice on your contract, a mortgage and a car that you believe impresses you neighbours then just imagine how long your savings would last to maintain your current quality of life.
I would be interested to hear how you would "adjust your lifestyle" in such a predicament and how you respond to hitting the advance button on your heating and not having enough credit to keep your heating on.
So your closing statement is valid, get your coat, put it on and stop preaching.
I have a theory for you, there was an on-line campaign for people to mass move their cash out of the banks and into credit unions on the 5th. The system outage means that anyone that had any ideas of moving their cash from RBS/Natwest would have been unable to thus defusing a perceived potential bank run.
Whenever I read of a spokesman announcing that they are "working hard to..." I dump the statement in the same bin that I would use for "lessons will be learned..."
"Working hard" is a meaningless annoying statement. I'm sure the technicians dealing with the problem have decided to go home, put their feet up and deal with it on Monday - NOT.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
