
"Millennium bugs"
I know El Reg likes to have fun with their headlines but I actually thought it was the millennium bug at fault here until I got half-way through the article.
The London Stock Exchange suffered a second day of problems yesterday as its new trading platform struggled to function again. On Tuesday the market failed to close correctly, causing confusion over closing prices for brokers and traders. Yesterday the MillenniumIT system displayed zeroes against some bid and ask prices, …
Since when has SUSE Linux Enterprise Server been equivalent to Microsoft .NET framework? Will El Reg be telling us that a new car based on the Internal Combustion Engine is the equivalent to a new car based on the VW Golf platform next?
You are supposed to be an IT rag after all...
*nix based systems are not real-time systems. Well, not if you've actually used a real-time system. Yes you can make them *look* like real-time if you throw enough horsepower at them but it's rather like watching that guy with the spinning plates ... as the number of plates rises ... he's going to start dropping them.
Royal Bank of Scotland is about to replace a whole load of its Infrastructure engineers with guys in India. Over the next year or so 10s of thousands of man-years of experience will be made redundant in the UK and picked up in India.
It'll be interesting to see how that experience goes for them...
People who bad-mouth money don't get what it is. You can't have lives without it. It is how resources get where they are needed and how injustices are healed. There are also a lot of sharp types who are constantly looking for a way to divert some of the flow into their own pockets- that's how we got flush toilets and mobile phones.
You think you could be happy without thinking about money? Stop thinking about it then. Let us know how you get on.
Lighten up a bit!
I think the original poster was making a thing called a JOKE. You see a JOKE is a funny thing people say to provoke a reaction to a situation that is becoming tense. JOKEs or HUMOUR can relieve the stress and tension and allow people to relax a little!
Now why don't you relax a little and take the comment in the vein it was so obviously intended you muppet!
Google it...Money is not the root of all evil...of course...but the antagonistic competition for profit is...it pits me against you...money is only the oil...still, we can, as a species, live without either...You lot have just not worked it out yet..one day :)
Time for the brightest minds on the planet to waken up I'd say...
If it was necessary a workaround would be found just as they did in this case. Critical bits would continue. 'Keep Calm and Carry on"!
Don't even consider talking about real time systems and their relative structure and merits until you've considered something 'mission critical' like train or EMU control, nuclear station or explosive chemical reactor plant control.... That is real 'real time' control. Everything else is just control. I suppose timeliness of information in the stock exchange starts to become important were you have automatic trades happening - but that is just automated greed...
Obligatory comment:
I seem to recall loads of people slagging off Windows, when the previous system was having problems, saying how Windows should never be used for mission critical etc. etc. etc. Now I presume that we'll see similar people saying the same about Linux...
In reality, hows about this:
Both OSes are perfectly capable of running high up time/high load systems provided:
1) The software is designed and written properly
2) The OS builds are competently designed and implemented
3) The infrastructure is designed and implemented properly
4) The operators are competent
And finally as a tedious PS, to show I've not left anyone out:
Why aren't they using mainframe?
Trading systems require real time features - that's the "low latency" bit. .NET runs on top of the CLR, which isolates it from the critical timing facilities it needs. Java would have been another awful choice for similar reasons.
The SUSE system will be based on the Linux kernel's real time extensions, and therefore makes much more sense as a platform.
so presumably you can point us in the direction of some metrics to prove your point?
The JIT compiler only compiles CLR code into native code ONCE, then simply does an address lookup from then on in. It may be very, very slightly slower on the first run of stuff, but I very much doubt there's a critical difference in the timings (unless there's a difference in resolution of the timings, .NET is limited to 10ms IIRC, but I've run tests on it before and it has performed very well). If you have some figures to back it up, instead of making assumptions, I'll happily apologise and retract. I also can't speak for Java, but I would expect that it does suffer from some slowdown on real time-critical systems because that extra layer does actually exist.
> so presumably you can point us in the direction of some metrics to prove your point?
It doesn't require metrics as such, just a better understanding. The issue isn't about a "slowdown" it's about a "slowdown just when you don't want it." You're not quite right about the JIT - although it does only do it's thing once, it doesn't do it all in advance. It does it when it first encounters a new code path, which can happen at arbitrary times. Same with garbage collection - as described here:
http://msdn.microsoft.com/en-us/magazine/bb985011.aspx
"When a collection starts, the collector suspends all application threads..." If this happens just when you don't want it to, you get a delay and the trade happens outside the required time window. Things like memory paging also cause little hiccups. You won't notice them except in timing critical applications.
In Java-land there's the Realtime Specification for Java modification which solves some of these problems (ahead of time JIT, controllable GC, pinned memory, etc). Maybe there's something similar in .NET-land?
Trading systems don't need real time facilities, what they need is to be sized correctly to process X volume of transactions in Y secs as general load, allowing for peaks, which will also be specified. This is not realtime processing. The actual response times don't seem to have been the problem with the old or new systems, rather the availability.
"But that was blamed on "human error", which the exchange claimed was likely sabotage." ..... Is there any other kind of malfunction if we are not to assume that there is mechanical life too?
*And their Stealthy TakeOver of Systems and MakeOver of Humanity is Sublimely Supplied by your Thinking that such Thinking, and therefore Life Phorm, is Preposterous and Most Unlikely even whenever IT Floats the Program in front of you.
"Wow .. I actually got to the second paragraph before I realised this comment was from amanfromMars 1" .... Anonymous Coward Posted Thursday 17th February 2011 12:54 GMT
I think that then positively proves the second paragraph rather nicely, AC. Thanks for the confirmation of the efficacy of the methodology.
I like to think that you have been advancing, AC, rather than suggest that things are being made much easier for more general understanding and virtual comprehension, although, as is quite natural and therefore to be fully expected in the quantum communications world, is it probably a bit of one and a lot of the other, for travel towards the Singularity of Omniscience Shared for a Better Beta Operating System in Big AI Societies and SMARTer Enabled Civilizations.
Why are we suddenly blaming SuSE or Windows for this? Do we know that this is an underlying OS fault? The article doesn't say. Just because I can't get OpenOffice to run on my own main Linux system, I'm not throwing out the OS, any more than I would throw out my Windows 7 system because it doesn't run Pinnacle Studio 10. Neither is the fault of the OS (in one case it's a setting problem, in the other it's the age of the application).
I'd want to see a lot more information about the cause and effect of this fault before I'd pass judgement.
When Windows was the OS hosting the platform lots of "the usual suspects" were queuing up to blame Windows/MS. Now there are a few people blaming Linux, but really it seems to be apparent that there is nothing like enough information available to blame either. In reality it's the infrastructure, support and software that are far more likely to blame.
"let's shut it down for a week or two .... see if we can't get on with our lives without worrying about money every second of the day.
maybe people would even find the time to be happy again." ..... Anonymous Coward Posted Thursday 17th February 2011 11:03 GMT
Please let it be wwwidely known, that that is the present abiding problem for the likes of the Markets, which isn't going away and grows ever bigger and more strident every day.
In a nutshell it is thus and this ..... There are those who are Ostensibly Anonymous who have hacked the money system protocols and would share what they have discovered, which removes forever the worry of not having enough money for one's needs. And of course, that then removes the need for those who would control currency supply, and really fcuks up their virtual command and control game, which has them living in luxury for free, off the backs of those who are providing them with their needs and feeds and seeds.
Now who would have that information and intelligence and Most Valuable Intellectual Property, is something which ensures the mystery insures and assures its holders and stock stakeholders, and thus is necessarily invariably always generally unknown as it is known and passed through Networks to SMARTer Keyholders and Trigger Guards for further Sleeper Awakenings for the Additional Protection of Greater Force Numbers.
The London stock exchange system used to run nice & reliably on VMS until Accenture took over and no doubt charged an arm and a leg to move it all to a "more modern platform". IIRC they moved or sacked all the LSE employees who knew how the existing system worked and then said it had become uneconomic to maintain. Management consultancy 101.
I understand the ball that goes up at the start of trading is controlled by Ubuntu. I guess that's a form of real-time system if it has knack all else to do but at least once it failed due to a full log partition/poor build. Funny how often that happens. Frankly for my money any financial trading system should be on Solaris with ZFS but what do I know, I never worked at any exchange and was too stupid to bother learning mainframe when I had the chance.
- Paris... because she knows where the money is.
Let's try unscrambling some of it
*Millennium bug". Failure cased by incorrect processing of change from 1999 to 2000. The article shows *no* evidence for this class of bugs being the cause of the problem.
Business critical. System fails company goes down pan. That *would* have described any major *offline* accounts app EG LEO (Lyons electronic Office). More recently would be the code that did calculations for a medical radiation source to decide, given various factors how much to dose them with. Not real time at all (except in the loosest sense) but *absolutely* critical to delivering the machines function. The code had bugs and several patients got 10x the dose they should have had.
Latency tolerant soft real time. Describes lots of stuff on *any* GUI. You clicked on "My Documents" and it showed you the folder. It opened fast *enough* that you're not bothered. If it takes a *bit* more (or less) time to act you're *still* not bothered. The cursor followed you mouse *well* enough to keep you feeling in control.
Latency intolerant soft real time. Here the *consistency* of the response is the important thing but I can't think of a situation where this applies.
After this things get *tough*. Typically the computer system is connected to "stuff" and if the computer can't work out the *right* response in a *guaranteed* amount of time *every* time *very* bad things happen. The "stuff" might be another computer on the network (Skype's latency and *consistency* of that latency are both pretty important for it to work well). If the system is an airliner, jet engine or Uranium enrichment centrifuge things can go badly wrong very quickly.
*nix's were never designed to support *hard* real time constraints but various upgrades to standard functions and the *long* background in building architectures that run such tasks have built up a *lot* of expertise in what to do and how to do it.
Armadillo Aerospace runs such a box to to fly the lunar lander which won the NASA challenge prize.
Some of the usual features for the work include a high resolution clock for the scheduler, software modules locked in main memory (no writing to disk) and either locking module scheduling priorities *permanently* or only allowing them to be changed under *very* carefully controlled criteria.
The developer and admin mindset is important here. While it looks to the apps and the dev tools like a normal *nix box it is not *run* like one.
Of course while the knowledge is out there and developers *with* the knowledge are out there as well it's not clear if *these* developers have it.
BTW Be *very* careful of the phrase "real time." Real time relative to *what*? A human can't control an unstable aircraft without computer assistance (watch the control surfaces of an F16 if you ever see one but *can* respond fast enough to simulate an ABS (c5 brake presses a second IIRC). OTOH a chemical plant with a time constant of 1 hour can be controlled by a human with a chemistry set, some graph paper and a calculator in "real time."
That's my pedantry done.
"Latency intolerant soft real time. Here the *consistency* of the response is the important thing but I can't think of a situation where this applies."
Flight simulator or similar? It's not necessarily a disaster if you miss a deadline slightly occasionally, it is useless if things are sufficiently unpredictable as to make them unrepresentative.
"*nix's were never designed to support *hard* real time constraints"
In general true, but (if you'll pardon the expression) there were exceptions of various kinds. DEC OSF/1 was one I was familiar with, though again it rather depends on what is meant by "hard". You wouldn't fly an aircraft, you might run an air traffic management system (and indeed Eurocontrol did).
I gave suse version anything a wide berth long ago, just after the time they sold out to those Novel scum. SDE was the most miserable pile of crud I ever had the misfortune to install on my machine, possibly even including Corel linux. Trying to run a business with SUSE you need to have your head slapped and to have your servers go titsup. Well, look what just happened! And this is me, the biggest linux fanboi since 1991. Serve them glad.
How do I make this FAIL icon bigger?
Don't use it. Period. Instead, use something like QNX Neutrino or RTEMS which layers a POSIX API over a hard real-time core to allow normal 'nix functions to run when desired (such as during development) but also provides the hard real-time API's necessary for time-critical tasks.
Use Ada, too, so somebody can maintain the bloody code 10 years after you've departed the project.
Mine's the one with the QNX demo floppy in the pocket.