The shocking part..
The shocking part of this update was the lack of a need for it.
There was practically nothing in it.
You only push updates if they are REALLY necessary and as you point out, you'd better be bloody sure that it is OK.
Not to heap more misery upon Nokia employees, but if you are a depressed Finnish employee, we recommend you stop reading now and turn to this adorable story about a rabbit adopting some kittens. What you are about to read throws Elop's challenge to the company into a whole new light. Earlier this week Microsoft issued, and …
Even Openmoko managed that one FFS! Stick a recovery bootloader/firmware updater somewhere that can't be overwritten(*). Updating over USB and the cable falls out half way through? No worries, hit the button sequence for recovery mode and start again.
* Ok, it can, but you need to take the phone apart and attach a JTAG debugger to do it. And if the user can do that they probably don't need to worry about bricking in the first place.
Tthat assumes the recovery bootloader actually does what it's supposed to do. Such a rarely touched feature could easily be DOA with no one noticing or bitrot over time, especially if the firmware format changed, or the signing key, or flash partitions got shifted around during a product's lifetime.
It may even be that even with a recovery mode, that the mode is not visible to the user. Or it may be you're meant to stick a pin in a hole while rebooting to retrigger it. Things that might be non-obvious to a user and require a return to store.
Anyway I suspect Microsoft's problem is the more phones they have, the more headaches they'll have trying to coordinate an update. Are they going to do a big bang update of every phone model, on every network or a rolling barrage based on phones coming out of the QA process. And what does that mean if phones later in QA expose bugs that could affect other phones. Is there going to be a second round of updates.
Having implemented firmware upgrade for an embedded device (it wasn't a phone, but the principle is the same), I can tell you precisely how you make this robust.
Not so much a recovery boot-loader, but a boot-loader that is capable of performing recovery when the usual boot image fails. It doesn't need to do much, and you should never need to upgrade it. In our case, the boot-loader was a small application statically linked to a heavily stripped-down kernel and device drivers. I think we got all of this in about 250KB, including full network stack.
The worst-case recovery scenario is that the flash filing system you are about to boot off is hosed, so the bootloader must be able to integrity check it make a judgement call to reformat and recover.
You either lock the flash sectors containing the image, or for maximum security, use a completely different storage device which is either one-time programmable, or which cannot be programmed in circuit. We used the first of these two option, and for good measure ensured that the boot sector was mapped into read-only pages in the kernel as well as using the processor's write inhibit for the flash-bus.
In case you wonder how the boot device is programmed in the first place, it's either via TJAG, or you use the services of a bulk programming service which tapes and reels the parts for assembly.
So, with this in mind, we can only assume a massive design cock-up, involving one or more of the following:
1) First stage bootloader not capable of recovery
2) First stage bootloader not write protected (and something accidentally overwrote it)
3) Failed Attempt to upgrade the first-stage bootloader
As with the previous poster, I suspect that the problem is the lack of recovery tool that is capable of being used by the customer. Writing one of these is not terribly hard.
...but then you could try hacks, break the phone, recover, repeat and eventually get access. Then you could pirate stuff (SHOCK!)
Make it do things MS didn't think of (HORRORS!)
Not have to upgrade so often (CALAMITIES!)
The world would end I tells yah!
The mere fact you could use such a system to reverse out a cock-up such as this is of no consequence to MS and they hold all the cards. They ruined PCs and they seem poised to do the do the same again on phones. It is a real shame. After netbooks, smartphones was the last chance for F/OSS. Unless they pull something out of the hat quick, they'll fail there too.
When you develop a BSP (board support package) for a mobile phone, it is your responsibility to implement the boot loader and the recovery mechanism for it.
Step 1: define a region for the main boot loader for the device
Step 2: define a region for the "shit hits the fan" boot loader
Step 3: Append a checksum/crc for each of these regions
Then you make a really really simply boot loader which does the following :
1) If no buttons are pressed by the user tries to verify the main boot loader and if it passes CRC check for that region, execute the main boot loader. That boot loader should also be able to recover the device if the "shit hits the fan" bootloader fails CRC.
2) If the user is pressing a specific button combination OR the main bootloader fails CRC check, then execute the "shit hits the fan" bootloader which contains enough code to restore the device via a USB connection.
If nothing else works on the device, this needs to be flawless. The program code to make this work is insanely simplistic and any idiot can program it if they think of it.
If you happen to be Samsung on the other hand, then you make your own boot loader which doesn't take the possibility of data loss or the need to replace either of the boot loaders into effect. As a result, the first time there's a failure, your phone is bricked and requires JTAG programming of the Flash memory which is why on new phone models, it's critical to expose the JTAG port of the device for programming through a docking connector or easily accessed test pins. This way, people who mail ordered the phone and can't get to a service center easily can be a bit enterprising, build a $10 cable and fix it themselves.
This screw up WAS NOT ... I repeat WAS NOT Microsoft's fault. The device should have been recoverable by the user NO MATTER WHAT! The developer of the board support package is the one at fault here and even if it was a subcontractor from Microsoft who made it for Samsung, it was still Samsung's fault for not having verified that the BSP functioned as defined.
My experience with working with Nokia on similar projects is... this would NOT happen to them. They are the most anal retentive assholes regarding these issues and they know to test this thoroughly before it ever leaves the factory. If you think you can just pay someone to design a phone for you and not bother testing it before it ships, you shouldn't be trusted in this business.
This sounds like a major problem; one of the things I liked about the old "Pocket PC" (Windows CE) was that it was almost impossible to brick.
I gave my old Dell Axim X50v to my twelve year old son, and he's changed the O/S on it several times (Pocket PC 2003 all the way up to Windows Mobile 6.5 and back) without bricking it. He even had a few problems along the way, but was able to recover from them and get back to a working device.
Firmware updates have been happening for years and years on smartphones, and OTA updates are just firmware updates over 3G/wifi rather than over a cable. So Microsoft screwed up with one? This may not be unheard of but it's generally quite rare and certainly not an indication of deep problems with Microsoft's OS.
I also fail to see how a Nokia bureacracy has any relevance at all.
Nokia will become just another Windows Mobile OEM, and will be running Microsoft's software like everyone else - that's their problem, not some imaginary firmware update methodology flaw.
If the Nokia-Microsoft deal amounts to a "special relationship" significantly above that which HTC, Samsung et al have with Microsoft (and I've read hints that Nokia will be able to customise WP7) then in theory they'll be able to have better control over the update testing.
Even iPhone updates cause problems occasionally, but the close tie between hardware and software definitely helps. Hopefully NoWin can do that too.
the Nokia-Microsoft deal does amount to a "special relationship".
Elop IMHO has huge conflict of interest. Check the "Top 10 Other Holders: MSFT" here:
Ownership Breakdown: MSFT
as of 2/24/11
1. Gates III (William H)
2. Ballmer (Steven A)
3. Marquardt (David F)
4. Daiwa Securities Group Inc
5. Bach (Robert J)
6. Turner (Brian Kevin)
7. Hastings (Reed)
8. Elop (Stephen A) <-----
You cant tell me that hes not throwing all of nokias investment in linux in the bin and going microsoft because its the best thing for the company, when he personally stands to make a small fortune on his microsoft shares.
My last mate who still has a nokia is about to go samsung galaxy s. The linux platform on the n900 was really good and on time. How can this happen?
What an utter load of misinformation in this article - please get the facts right before you attempt some Microsoft Bashing.
>>Earlier this week Microsoft issued, and then withdrew, a software update for its Windows Phone devices.
They've suspended the update for samsung only phones!
>>It's a nightmare for the manufacturer, because it requires an expensive return to a repair centre, and the customer, once burned, is much less likely to become a repeat purchaser. That "small number" may be as high as ten per cent.
Yes 10% of updates failed, but over half of the updates were caused by a lack of disk space or a poor internet connection. The update process backs before patching, and a lack of space to back up will prevent the update from taking place - though this doesn't brick the device.
>>Bizarrely, the patch didn't contain any new features. It was a notification about new features to come, the equivalent of that little alert you get in XP telling you that there's a new version of Software Update available
The patch was an update to the updating process - not a notification, and was probably purposely done this was as a trial roll out - which is another reason the patch is being rolled out slowly rather than a big bang approach. Surely it's better to roll out a small update to test the process first before a major update in march!
>> OTA - Over the air updates
You only get a notification an update is available over the air - the update is performed via a fixed connection through zune of the mac windows 7 phone connector, not over the air.
10% is a huge percentage. It's like a 10th!
These are phones that have to be fixed by the phone manufacturer that installed the WP7 badness. Their cost not Microsoft's unless Sammy ask for their money back.
This was a massive fail.
To say otherwise is to try and spread Harmony. Only someone paid to do so would do so.
It was indeed a massive fail...of Samsung's.
The most reported cause of the truly troubled phones (not those that just failed to update, and were fixed by powering off and back on - which was the most common problem) by those who actually studied the fault have all been pointing towards certain firmware versions for the devices...particularly those ones designed for the mobile networks specifications - like T-Mobile UK, whose bootloader is locked to make sure you can't update it yourself, you have to take it to someone with the right jig (I built one), ie a repair centre or store, and pay money to get them to do the update.
My Omnia 7 updated without the slightest hitch because I updated my firmware to a non-T-Mobile branded proper version.
Nokia can update the same OS over what must be around a hundred different models/form factors, each one with a standard profile or with operator customisations, different regions/languages, etc...
The fact that WP7 can't cope with this isn't Samsung's fault. They did, after all, customise their phones' firmware within what MS allows (what little they let manufacturers customise) and even then it blows up.
Look, there are obviously reasons for why the update failed. That doesnt excuse them happening though. If there wasn't space to backup files, some kind of "I don't have enough space to proceed" message is required. If the internet connection broke halfway through or the firmware was corrupted, then why wasn't the file signed so it could be checksummed?
It isn't acceptable to brick phones except in the rarest of circumstances. Even there it would make sense to provide a mode for the user to be able to recover from that state without an expensive return.
"But it does shed light on the magnitude of the task Nokia's new CEO Stephen Elop has set the company. I'm not sure the risk has been fully factored in by analysts"
I don't think Elop really cares. If Nokia succeeds at selling Windows phones then great. If the company fails, then MS will buy them out ala Novell.
"I don't think Elop really cares. If Nokia succeeds at selling Windows phones then great. If the company fails, then MS will buy them out ala Novell."
Indeed. MS get their grubby mitts on "intellectual property" to extort cash from others and threaten the competition, whichever way it goes. Anyone who flushes the high-margin product pipeline in such an apparently cavalier fashion and trashes the company's publicly-estimated value is either clueless or wants to bring the company to a state of desperation where an acquisition just looks easier for everyone involved.
And that wouldn't be the first time Elop has been involved in getting a company acquired.
No need for twisting - it's just that MS rarely manages to look good, if you take an objective view. It's a huge company with massive resources and no vision, that consistently delivers poorly tested incompatible software, behind schedule and usually by imitating another developer's product.
...but I am the proud owner of two bricked Windows Mobile phones (Motorola Q and a Samsung Ace) that my 2 year old loves playing with, and I'm currently using an HTC Touch Pro 2 that I'd throw into the toilet if my company wouldn't just give me another one of the same.
I've used Windows phones for the last 6 years and am quite content with the move from XP to W7 - but trust me, you don't have to twist this or be a Mac/Linux fanboi for this to raise significant concern.
that customers of mobile phones have more choice today than those available to desktop and laptop buyers in the past. The one big thing going in Microsoft's favour is that most consumers have very short memories and don't give a fig for the technicalities - they just want new, hip and shiny. (By "most" I do not include the denizens of this forum of course).
Looks like the full desktop experience to me, slow boot up, plenty of crashes and updates making a mess of stuff that was working before.
I dare say if anybody manages to keep their phone for a year you'll find it running at 1/2 of the speed of when you got it and wondering where the bloody hell all the disk space has gone and wishing you had bought something else.
Oh, and it will be full of viruses just by a website drive by....
Mines the one with the android in the pocket....
Engaget had this story 2 days ago. It only effects a few Samsung handsets with specific firmware revisions works fine with other vendors. The update isn't just a notification it updates the update code.
The so called 'bricking' of the device can apparently be fixed by removeing the battery, putting it back in and turning on the phone.
OK, setup. N8, creaky old Ubuntu/XP-2 IBM T30 dual-boot laptop, which has been staggeringly reliable over the last, oh, 8 years or so. Mobile internet connection (no, it loads onto the PC first, _then_ transfers, so no worries).
I decided to upgrade because when I tried to put the music player as a main panel shortcut, it insisted in putting the radio there instead. I had two radios on the main panel.
OK, go S/W update. Yep, update available, but at 152 megs, only way is through OVI Suite.
Link all together, go OVI, and find there's : Updates for OVI, N8, and some apps.
Now, this *might* be a bit of my fault, as the lappie only has 256M of ram. (Stop sniggering - I only use it in XP mode for OVI) As a result, this Alzeimer's suffering 'device' takes a full 20-30 minutes to stop thrashing its disk on startup, with NO apps running. Not that I can get an app running in that time, the mouse cursor is as nimble as a slug. Ubuntu 10.10, it thrashes 2 minutes.
Offered the choices:
1) update OVI
2) Update N8
3) update apps.
Fearful I'd need the cursed "Restart Windows to complete..." stuff, I (foolishly?) decided to skip 1) as I wanted to do something else with my life that evening. After all, the version I had worked, it's only an upgrade/reflash innit?
So, choose 2) and start the 1 hour download. Download OK. Start the upgrade to the N8. Numerous pop-up messages telling me "This device can perform better if connected to a USB 2.0 port. Click here for a list of available ports, which instantly gives the message "there are no availble 2,0 ports".
Aside - Why the Fuc*king hell couln*t the stupid twat who wrote that bit of code not have got it to check first" Alice-in-Wonderland "It's rude to offer what you don't have". I digress.
OVI lied. This process will take about 15 minutes. Who'se minutes? Anyway 3/4 of the way through = ½ hour, while the N8 is now displaying 'test mode', OVI tells me there's been a problem, disconnect the USB, reconnect, and click "here". OK, Mummy knows best, so I did. B'stard started from the beginning again. Did it twice more, at which point I'd had enough, take it back to the shop, and they can reflash the fuc*ker. All I had was a white screen with NOKIA written on it.
Turn off, turn on, and - I'll be jiggered, it groaned into life, with umpteen messages. The bug I started this process for is now fixed beautifully. Does seem more responsive, too.
Daft thing is, I did upgrade OVI afterwards, and - it doesn't require a reboot.
Start all above at 3:30, finally finish (ok, loaded latest maps for Finland afterwards) at 9:00. B'stard made me miss 'happy hour'.
An aside, I went to the Nokia.fi site last night. After a few seconds, this big blue banner appears. Clicking 'Sulje" - close didn't. Goto Nokia.com. Same banner, went away obediently on clicking 'close'.
Try with Firefox, no problems.
Now, as it only fucks up in the Nokia.fi site*, and only with Explorer, do you think Elop is trying to tell the Finnish Nokians something...
*I expect it's been fixed now.
Sorry, can't remember what version of explorer I have, and I can't be arsed to endure ½ hour of disk-thrashing to find out, nor go down 4 flights of stairs to retrieve the T30's bits from the car park, and put it back together again...(joke - it's only 3 flights...)
Mat, 'cos it kinda looks like a brick.
So what you are trying to say is that you ignored all the instructions and minimum machine specs and are now upset that the update did not go smoothly? Nokia software may be shite, but for this one I think you only have yourself to blame. On a plus note, an OTA update via wireless would have saved you a lot of hassle.
Nope. Not available. Commands me to go to OVI suite. Not sure minimum machine specs come into this with only one prog. running. Ovi does everything else. You're probably correct that I should've upgraded OVI first (it only took 2 hours, but no reboots), as it tells that this fixes certain connection problems between PC and N8. But I'm buggered if I'll do another upgrade on this setup.
Assuming there ever will be one.
The bozo HW/Cost-cutting dept. at Nokia decided to scrimp and save and give 256 megs. for system memory. OK, oodles for porn^Wmedia and apps. in the main memory, but... there's no room for a fallback if the download is 157 smegs and goes tits-up. Assuming the "E:drive" isn't used as a temp.
Jeepers, wouldn't have added more than a dollar to the build price to double it. 50c. if I were the buyer...No bricks!
In their unimaginative, clunky way (let's leave their business ethics for another day) But Nokia can't do software at all. They bought Symbian and destroyed it. They bought Trolltech and were about to finish it as well... Elop is a trojan horse, but one that the Nokia board and the previous boss Jorma Ollila brought in knowing very well what they were getting.
A botched update was expected of course, and up to 10 percent failures? Delightful. To blame both the users and the handset makers was a stroke of genius. Over the top. Please wait 4-6 weeks for a direct ship replacement for your phone? A work of art.
A shame there's no way to way to blame the carriers as well for the trifecta. That would have been perfect. Hopefully there's a way to work that into the main update.
Not that this update bricked some phones but the reports that Microsoft and Samsung are passing the buck on getting the bricked phones sorted.
One would imagine that from the get go there should have been clear lines of responsibility defined by the licensing and related agreements between Microsoft and Samsung.
Epic fail on both from both of them.
it's a beginner electronic project to make a non-solder (ie: clip-on) jtag connector to allow unbricking of devices. I've made one for linksys routers, for instance. Alternatively, a little bit of ROM on the system that contains a base, working version of the firmware could be used for recovery.
For wireless firmware updates, all it would take is to first download the update into a dedicated bit of RAM, THEN do the swap internally. It's not exactly bloody rocket science here.
Modern hardware being irretrievably bricked is the sign of someone cheaping out on the design. Someone, probably an accountant, decided that they just didn't give a shit about the consumer. It's not necessary, it's wrong, and anyone buying a device that CAN be irretrievably bricked should reconsider buying that device.
First off, why isn't this story titled: "Samsung Fails to Properly Implement Yet Another OS". After their dumb filesystem that crippled Android on the Galaxy S, they've done it again, but this time to Windows Phone.
Blame Microsoft and get some phony kudos if you want, but in this case it really wasn't their fault if their OEM hasn't the technical ability to bring up the OS properly on their hardware.
Now, that's the spin out of the way, what about the revisionism. In the potted history of Apple, this paragraph in particular made me read twice:
"When Steve Jobs carried out his reverse-takeover of Apple, he brought with him staff experienced in creating high quality hardware, and mature, proven software, along with the people who had written it. They had experience of working together. That "skunkworks" was the NeXT."
After the words "reverse-takeover" the reality content drops away rapidly... NeXT had no hardware engineering to speak of by 1997, having moved to a software-only business, and an x86 one at that; but even ignoring this, the parts of OSX that made it "OS X", rather than just "NeXTStep PPC" were pretty much all developed by pre-existing Apple engineers. Finder, Core Graphics, Quartz, QuickTime, Apple Type Services, AudioKit, Dock, HFS+ were all "old Apple" components. So, hardly a takeover of software, either.
The big difference post-NeXT wasn't the engineering talent, but the focused, single-minded management that came straight from Steve's office in IL1. Gil Amelio wasn't the type to sit in on software dev meetings; Steve was there all the time, and on other groups too. Basically, if the user was going to see it, Steve was involved in it -- and most of the visible parts of the OS were produced by old Apple people, not NeXT people. This is what made OS X a success - there was one person who knew what was going on all across the development, and he was also the CEO (Interim).
Yes, the NeXT guys ended up promoted over Apple folk, but that's the way of it; they were seen to have rescued the company -- but it wasn't their technology that saved Apple. Apple already had technology coming out their ears. It was application of that technology to something a customer might want to buy that saved Apple.
While the glitch that caused bricked Windows Mobile phones grabbed headlines, what also caught my eye is the 10-step cable-based updating process that Windows Phone users have to slog through to update their phones.
There's a much easier way to do it. For several years, many mobile phone manufacturers and operators have been cutting the cable and using FOTA (firmware-over-the-air) software to update consumers' handsets. This method greatly simplifies the process of receiving and installing updates on mobile devices. Every day, millions of mobile consumers get software updates over the air-easily, quickly, reliably, and without fuss - thanks to FOTA. No PC or cable required.
To those last remaining companies still using cable-based updating I say - catch up! It's time to make the mobile experience truly mobile and unplug the cable.
is "who do you trust"? Or even, "who gets to do the trusting?"
I don't really trust the operator whose sim happens to be in my phone. But then I make sure the handset is all mine; I'm interested in them as a carrier of voice and maybe bits, not as upseller of things I don't need. And yes, mine's a PAYG. So I don't want them to deliver firmwares to my handset. And then, I might as well use the charger cable to put some open source firmware on my phone.
Which incidentally is what got me interested in symbian, only to find out that its open source-ness was mostly a lie. I mean, a hundred million deployed handsets and the stuff that gets opened runs on exactly none of those. No hardware support. In the land of winning hearts and minds, that's a typical foreigner's faux pas.
Of course, OTA updating is great for those cases where the handset is more an extension to the operator's network, so it's useful to be able to. But then you still don't want the thing to brick.
I'll lift a nice choice quote from http://www.h-online.com/open/features/Nokia-and-open-source-a-trial-by-fire-1194928.html?view=print :
``[T]he Maemo community was [...] asked to switch from Gtk to Qt, and then to merge Maemo with Moblin to form MeeGo, and then, on the eve of launch late last year was asked to re-write again, this time using QML instead of custom Qt based widgets and the impetus was lost.''
Symbian nee epoc and a multitude of UIs with lots of murky politics and backstabbing attached was discussed earlier here on el reg. You know who also had to go through a similar ordeal, with even more spectacularly withering results? That's right, Danger after redmond acquisition.
I am similarly impressed by how redmond's "7.0" compared to its 6 series is in reality another "1.0" and of the usual micros~1-"1.0" quality to boot. Yes I bet it's pretty nice, if you overlook details like how the rest of the world has had nicer stuff for years now.
That fits well with this: So it was an update to the updating process. That's pretty much "nothing much in there at all" as far as the custome^Wuser cares. To have that go kaboom is fairly impressive, even for micros~1. But "excusable" if you accept that their 1.0 is really only alpha quality. Even if it's labeled "7.0".
I don't know if a large bureaucracy would by definition help for Q&A. nokia and redmond both are large and bureaucratic and both have trouble with the quality of their shipping products. The thing about two turkeys comes to mind again.
I think the ability to stick to an internal acceptance process is more important, but for that you'd still have to have your priorities straight. And yes of course it's "on hold", whether it's true or not. Both from an operational PoV and for leaving room to spin it whichever way they want later.
It's true that large bureaucracies are very good at this stick-in-the-mud thing, especially compared to upstart start-ups, but that doesn't always align with great Q&A. In fact, I do recall a review of a nokia s/w update on el reg that was less than unanimously celebratory. But it's equally true nokia hasn't as publibly botched the updating thing. Yet, anyway.
We already knew nokia was in for quite a ride, this is just that much more icing. Doesn't surprise me much, seeing how I did mention I didn't make Elop out to be equipped for re-architecting the swamped-in-their-own-bureaucracy nokia. So to me it's not really ground-breaking news, but it does mesh well with previous analysis.
By the by, how much time would you reckon Elop has left to show or at least convince through smoke and mirrors, that his steering makes a change for the better?
Biting the hand that feeds IT © 1998–2021