Microsoft says axed certificates were FAILING its software biz Microsoft has admitted that the masters-level certifications it suddenly cancelled two weeks ago weren't delivering the skilled workforce the company needs to make its products a success in the enterprise. The admission came during a conference call, a recording of which The Register has obtained, staged to engage with those …
So a software company is not capable of grading an exam for a certification using software and has to do it pen and paper
I'm fair sure the CCIE is an all-day practical exam, and I seem to remember Novel's top NDS exam (which didn't last long) was performed on a collection of Netware servers.
At basic levels, sure, you can do computer based multi-choice/simulation testing. But at the more advanced levels, it's really hard to write a system which can grade an answer to the question: "Design an Exchange/Sharepoint/BizTalk/AD/wahtever system for..." which then has a page or two of requirements.
At the end of the day, that's the aim of these high-level certifications: To make sure people can design and build complex systems and there's rarely a straight Yes/No answer.
"At the end of the day, that's the aim of these high-level certifications: To make sure people can design and build complex systems and there's rarely a straight Yes/No answer.
Surely that is the challenge. As often as not, complex mutli factor analyses have a binary output - to invest, or not. To upgrade or buy new, To acquire or divest. Obviously the random strike rate on simple yes/no answers would be a problem, but by factoring in questions at decision points within case studies or problems, with mutliple possible answers, surely it is feasible to ask sufficient questions that the accumulative evidence enables you can sort the wheat from the chaff? Arguably that's all that happens at most interviews, and even then in an imprecise manner. As for "competency" interviews, what are they if not tick boxes?
How do I know that my CIO knows his stuff? Certainly isn't that he had to write an essay as part of his masters. Simply that faced with complex problems he has relevant knowledge to enable succinct and useful business actions or recommendations. If we're capable of having binary machines that can come within a country mile of pass the Turing test, surely the inability of exam setters to come up with machine markable answers is a reflection on them, not the concept?
Written exams all too often are simply an endurance test, serving best those who write fast and eloquently and have good recall - without doubt useful skills, but not necessarily the ones that the exam is supposed to be testing. I believe that it would be possible to set a paper with mutiple choice answers that only a very highly qualified technical professional could answer. There's no reason that you couldn't do the same for much of education, testing specifically for the skills you need, rather than using proxies like the quality of structured prose, or arithmetic ability.
Microsoft once followed up on my complaint about not being able to access a new, very-expensive, volume-licensing agreement for a school online, several weeks after purchase.
Turns out, that all the electronic-only forms I'd completed and electronically signed, etc. were then handed to someone who had MISTYPED the word "administrator" in the email address of our, well, administrator (me!).
I don't know what worried me more - the fact that somewhere along the line someone had to hand-type in all this electronic information that I'd provided into some other system to make things work, or that the guys managing the volume licensing access couldn't spell "administrator".
Uh, no.
Frankly, that's an idiotic and uninformed opinion. "Master" and "Architect" certs require a whole lot of dedication and understanding of the platforms - as much or more than any Cisco certification, as an example.
Anyone who thinks this is "learning by rote" has never tried it, has no clue of what they're talking about and should shut the fuck up.
Seriously Jake. Talk about what you know and don't talk about what you don't know. It'll help your karma no end.
So the man who has been advising the boards of Fortune 500 companies since the 80s (or was it 70s) has, until a couple of years ago, been taking Certification Exams which by his own reckoning are designed to allow numpties to pass exams by rote.
Seriously Jake, if you are going to insist on spouting bullshit at least make it semi believable.
Nobody takes the Architect or Masters Exams for fun and nobody stops taking them due to Financial pressures. For most people the barrier to entry is the sheer amount of time which needs to be invested, something your schedule of yoof camps and hand knapping micro breweries would leave you short of.
I didn't do it for fun. Frankly, it was fucking boring. I sat the exams because Mindless Mega-corp Management Morons expected to see them on my c.v. I made a boat-load of money as a result.
I did not learn a single thing from the "classes". Seriously, I've been in computers & networking for nearly forty years. I already groked the curriculum.
You trawling through my back-log of posts in a childish attempt at a piss-poor ad hominem tells most cognizant readers more about you than it does about me. Seems my time management skills actually work.
Never worked with one of these master level guys. But my experience with MS certified people is as follows: you get them to design something, paying a quite good sum in the process. The end result is a design that usually requires the latest versions of the greatest and more expensive MS technology, even if some components could be replaced by, say, open source pieces. But no, you have to go to the full stack if you want to have the perfect and trouble free setup.
Said design also has the interesting property of ignoring all technical infrastructure already in place, and never quite ends up interacting well with all the gear you already have in place. Of course, you'll never find out this until you actually try to deploy these technologies at large, and the answer will always be "works for me, but that's because I have the latest versions of everything" When you reject the idea of doing that just for the sake of the new component working, they'll start digging MSDN for patches, workarounds, and registry hacks, which they'll apply meticulously.
At some point in this process, something will break and they will not be able to repair it. This is the point where your certified MS person will give up and tell you to reinstall everything again. Just in case, because these things are so complex that no one really can understand what they have done.
After a while, when your project is already delayed and over budget, you'll have to accept reality and live with this sporadic login prompt appearing from nowhere, or with the need to reboot some server on weekends because what is running there leaks memory or resources, crashing after a while. Or with the occasional show stopper timeout. Or...
Wonder if these top level certifications are really teaching more than "use the latest versions of MS everything" and "how to configure SP to use forest domain delegation" Perhaps they are going a bit beyond that and explaining WHY they'd want to do those things in the first place.
Still not enough. Because the truly valuable professional is able to tell you also why YOU would want to do these things in YOUR context. And more importantly, WHY YOU NOT. Which is against the spirit of the training, after all you train these people to sell more MS software, not to advise a customer when it should not buy it.
So I guess the open question is... has one of these 200 top gun guys ever advised against using an MS product?
At a guess, you've been hiring in contractors to do this.
As a holder of various certs (though not the master-level ones), and having only been a full-time employee I've regularly recommended open-source or alternative solutions where appropriate. There's also some time required before any proposal to look at what the business actually needs (in one case I had to talk an employer out of deploying SharePoint and instead putting in Wordpress, as all they actually wanted was a corporate blogging system).
Contractors don't have the same perspective of a business, nor the same appreciation for how a business has developed in the past and should continue to develop taking account of what's currently in place. Getting someone who'll still be there after a system is deployed (if for no other reason than that they can be yelled at until they fix it if it's wrong) is vital. Contractors are suitable for short, completely standalone depoyments but for anything integrated you need a person who is actively involved in your business.
You are completely right, having things done by someone that has to stand there after something is deployed is the key. And using contractors is one of the ways of lowering the chance of that happening.
However, in my case it is even more contrived. Because they are not contractors but "partners", supposedly engaged in a long term relationship with their customers, win-win situations, creating synergies and all that fancy speech.
But that is only "true" at the "partner" level. At the individual level is not, because anyone working with our account rarely lasts more than six months: if the individuals are good, they are usually expensive (and onshore) and end up moved to more profitable accounts. If they are not good, it takes about that time for us to realize and tell the "partner" that we don't want them in our account, so they are gone as well. So we end up working either with good novices or bad apples, the first do their best but lack the experience to do a polished job, the latter are simply bad.
Think about that next time you're about to sign off an outsourcing contract.
This cuts both ways.
I've worked on an enterprised wide rebuild for the back office of an organisation, which had spent 18 months on a proof of concept - longer than it took us to build the MS-centric final product, because the architect refused to use any Microsoft products anywhere in the solution, and was struggling to make all the open source work together.
The guy in question was extremely smart and highly capable - some of you in the open source community would know him by name - but he allowed his prejudices to blind him, and the opensource version of the project failed.
The problem isn't Microsoft products, and the problem isn't open source. The problem is zealots who refuse to use a tool or insist upon its use, because it is or isn't made by Microsoft. Its just not professional, and our industry would be better served if the zealots find another line of work - Apple stores, for example.
When we look at products like Windows, Exchange and SharePoint, those are billion dollar businesses,” Sneath said. “We have less than 200 certified individuals for each product. 200 does not give us the volume to be successful deploying those products to the enterprise.”
Sooo.. no one will have able to successfully deploy these products now?
If they wanted to dumb down the exams why not do so gradually, rather than this disruptive approach which is alienating the very people they want to sit these things.
A long time ago I did some of the RedHat certification and got myself an RHCE. To do that required sitting in a classroom all day and fixing/configuring a PC to do various things. No notes were allowed and it was just you and the PC. The concept being that you had proven (under an NDA as well!) that you had the technical skills to get stuff done. I believe that some of the (higher?) Cisco certification also requires hands on.
I don't see the point of having paper-based or online (multiple choice?) questions for technical products when it comes to showing you know something. Surely the proof of your skill should be shown by actually doing something. I must admit when I looked at some of the Sun Solaris certification some time back it was based on mechanical learning and multiple choice qustions and I wasn't impressed.
Actually the entry-level Cisco exam for the CCNA cert has built-in simulators requiring a knowledge of the commands and understanding of the curriculum. We sent two of our guys out for 7-day boot camps and they crammed like mad and spent their evenings reviewing and practicing in the lab, and they both used most of the allotted time for their exam. They both passed, but both conceded it wasn't as easy as they thought. It's a good method, regurgitate some of the specs AND be able to do the real work.
I've heard the RHCE is similar, where there is a good chunk of practical work involved. Our head Linux guy did it and was really impressed with both the training and the certification program. Those sorts of certification programs are quite valid in my eyes and even though the CCNA is entry-level, if you've gone out and earned it then it's worth bragging about.
I've taken a bunch of MS exams but only remember one of them having a simulation exercise (which crashed). To be fair, my NetApp, EMC, HP, and VMware certs were all the same too. Might be a different story at higher levels though.
""I've taken a bunch of MS exams but only remember one of them having a simulation exercise (which crashed)"
Surely that is an accurate simulation of a Windows environment? :-)
Still enjoy seeing BSOD on cashpoints, airport information displays etc."
That was just part of the test to see if they knew ctrl-alt-del
I've seen some presentations and examples of content for this track some years ago and we (me and other attendants, all MCTs by the way) had the following reservations:
- 2 Weeks at Redmond for intense training (this has been dropped)
- It's ONLY valid for a specific version of the product/technology (Exchange 2010, Sharepoint 2007, SQL 2008)
Our big question mark was; is it worth the time/money and effort? Obviously people have spoken and the answer is clear; no it's not. It might be useful, but given the required investment and the fact that your efforts have to be recognized by potential customers, it doesn't make enough sense from a business perspective.
Spot on. If you're not already in the USA it is a big time and money commitment to get over there for the training - and then the course fee on top of that.
Then the exam/lab itself is very complex and there's a chance you won't pass - and there's no money back guarantee.
Its been year since I started to think that most of MS's certifications are meaningless; people are able to pass them using cheat sheets from the internet; I've even seen people who know the subject fairly well use cheat sheets because its easy and the tests don't always align with the real world.
The certifications need to have value, and much like the MVP program, that value needs to be value for MS, so we shouldn't expect much from either.
I just wanted to make a clarifying comment in regards to the particular certification program that was cancelled. Please don't confuse the MCM, etc. with general purpose Microsoft certs such as MCSE and MCITP.
The one was highly valuable and genuine to those that took it and the customers that cared (generally very large enterprises).
I took the training in 2006 at a cost of $50,000 to the company. It was $25,000 in tuition to attend the 6 week course in Redmond and the rest was the cost was allocated to my time to be there plus expenses. I was one of the top Exchange consultant for one of the world's largest tech firms....I really thought I knew my stuff going in having designed systems for hundreds of thousands of users. I was immediately humbled. Everyone in that class was an expert. They all seemed to know far more than I did. The content went right to the heart of the software...For example, we not only learned the nature of the B+ tree in the enormous Exchange database scheme, the caching mechanisms, etc...but we also spoke extensively with the product team to understand the rationale behind their design choices, the impacts of various external factors to performance & reliability, etc.
When I was done, I had a whole new understanding of the product and also came away with knowing key people inside of MS on a first name basis. So...for my particular job - it was well worth it.
Regarding the exams...this program had both written and lab based examinations. The final 'qual-lab' is an all day affair. Most candidates do not pass on their first attempt.
David
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
