Too late now for canary test updates, says pension fund suing CrowdStrike CrowdStrike, after suggesting canary testing as a way to ensure it avoids future blunders leading to global computer outages, has been sued in federal court by investors for not using a phased approach in rolling out updates to customers in the first place. In what will likely be one of many class-action complaints against the …
It's possible they had some original involvement, but for most shareholders the likelihood is that they didn't "invest", but bought someone else's lottery stake. It is possible for shareholders to influence the behaviour of companies, but most of them seem to find that too much of an effort.
And for that you got a downvote?
It's beyond belief that there are commentards who can't grasp that simple fact of company law. The word company refers to the company of people who have come together to own it by buying shares in it. Unfortunately some shareholders also seem to overlook this simple fact. Sue the manglement - that's reasonable - but otherwise congratulations to the lawyers who got the shareholders to pay them (the lawyers) to sue themselves (the shareholders) to maybe get some of their (the shareholders) own money less the cost of two sets of their own lawyers. Two sets? Of course, as plaintiffs they're paying to sue and as members of the company they're paying to defend themselves against themselves.
Doctor Syntax:
Not really. A certain LLM says "Shareholders do not "own the company" in the traditional sense of direct ownership of its assets and operations. Instead, they own shares, which represent a claim on a portion of the company's profits and, in some cases, a vote in company matters.
Here are the key aspects of what shareholders own and their rights:
Equity Ownership: Shareholders own equity in the company, meaning they have a financial stake in its success or failure. The value of their shares fluctuates with the company's performance.
Voting Rights: Typically, shareholders have the right to vote on important company matters, such as electing the board of directors, mergers, and other significant corporate actions.
Dividends: Shareholders may receive dividends, which are portions of the company's profits distributed to shareholders. However, dividend payments are not guaranteed and depend on the company's profitability and policies.
Residual Claim: In the event of liquidation, shareholders have a residual claim on the company's assets after all debts and other obligations have been paid. This means they are last in line to be paid.
Limited Liability: Shareholders are not personally liable for the company's debts and liabilities beyond their investment in the shares.
In summary, shareholders own shares in the company, which give them certain rights and a financial interest, but they do not own the company itself in the same way that an individual might own a private business. The company's management, under the oversight of the board of directors, is responsible for running the company and making operational decisions."
Me again: Said rights are stated in the prospectus (usually for new issues) or other definition/offer document. If the company fails in its duty to run the company properly and according to its share offer statement, it's perfectly valid, and a common thing, that shareholders sue the company for misrepresnetation, misconduct, whatever. Of course, such action will probably affect the share value, hence shooting shareholders' own feet.
Well I guess if an LLM says it, it must be true, right?
"Shareholders do not "own the company" in the traditional sense of direct ownership of its assets and operations."
This sounds like the LLM is answering a different question. If you asked whether the shareholders directly own the assets and operations of the company, then clearly the answer would be no. The company owns the assets and operations of the company. If the question is whether the shareholders own the company, then yes, the shareholders, collectively, own the company.
"they do not own the company itself in the same way that an individual might own a private business.
Surely an "individual who owns a private business" is a shareholder who holds 100% of the shares.
It's beyond belief that there are commentards who can't grasp that simple fact of company law.
Perhaps because it's by no means that simple?
"Own the company" is a largely meaningless phrase for a publicly-traded corporation, since "ownership" is a broad concept and only a handful of its possible aspects apply to shareholders (individually or en masse). So this objection to shareholder lawsuits is sophomoric at best.
More importantly, it doesn't make economic sense. There are at least two sets of conditions under which a shareholder lawsuit is potentially valuable to the plaintiffs. First, if the value of the company has been irreversibly damaged, then a shareholder lawsuit may extract more value from the failing firm, for the plaintiffs, than they'd get from stock ownership. It could establish the plaintiffs as preferred creditors who get a larger share of the proceeds when the corporation assets are liquidated. So it can be a hedge against complete loss in the event of failure.
Second, if a proper subset of shareholders sue and win, they may transfer wealth from other shareholders to themselves.
Only in the in simplest case but company's have liability insurance and they will pay out if the company loses?Rather than suing the company they are suing the insurance company.
Possibly not, or the insurers at least will try very hard to avoid any payout. It's possible to insure against fraud by employees, but harder with execs. Plus there are claims for gross negligence which generally are uninsurable. And this claim includes some of the execs, so possibly going after their DOI cover, but that again doesn't usually cover fraud or negligence. I get the feeling litigation is going to end up a very expensive way of answering a simple question "Did you test the release before deploying it? Yes/No?"
The original investors in your sense would have done soe in hope of a return, either by dividends or by being able to sell their share of the company to others. The existence of those willing to by shares in the after-market are ultimately responsible for the willingness of the original investors to invest at all. And they themselves are now investors because they now own a slice of the original investment and because they were prepared to invest in it. It's unfortunate that they haven't grasped the fact that, together with the other shareholders they are members of the company they're suing. They're suing themselves.
I suppose it's just possible that this is a vehicle to get some of their investment back before it's swallowed up by customers' suits.
It's unfortunate that they haven't grasped the fact that, together with the other shareholders they are members of the company they're suing.
I don't think that's really a problem. These shareholders are also the trustees of a pension fund, so have a fiduciary duty to the members of the Plymouth County Retirement Association. Plus this is a class action, so Plymouth County becomes the lead plaintiff in that action because it has standing, ie has been harmed by the $120-ish fall in share price. But having been through a few Ch.11s, twice as an employee (not my fault) and then as an advisor..
I think this is the usual defensive/opportunistic class action. Opportunistic because if it wins, the lawyers make bank. Then defensive because once the law suits start flying, like Delta's claim for $500m, the risk of Ch.11 bankruptcy proceedings increases. Then if CrowdStrike's forced into Ch.11, the ordinary shareholders are usually wiped out. So investors joining the class action might mean they get paid something because the litigation continues under Ch.11 proceedings, unless it's settled. Then if they win, the awards are protected and survive bankruptcy, or can get settled during those proceedings.
But as you say, it's one of those slightly strange things when the litigation can trigger Ch.11, so if shareholders don't join in, they risk losing their investment but if they do, they risk destroying their investment as well. Especially as shareholders also have a fiduciary duty to the companies they own. I think this is just one of those strange things about the Ch.11 process.
I don't claim to be an expert in investment law, especially in the USA, but this seems odd. What exactly are the shareholders' duties to the company?
Theoretically the same as any other fiduciary, ie act in the best interests of the company they own. From memory (and IANAL) there have been cases where shareholders have banded together and tried to vote changes that have benefitted those shareholders, but not the company or other shareholders. It's also where I think there can be problems with Ch.11. So the usual vulture capitalism, where debt holders may be secured creditors vs shareholders, but accumulate enough debt and trigger a default & Ch.11. Then shareholders get wiped, bond holders get their debt converted into new shares and take control of the company. Then often flog those shares as part of a merger and end up with more than they paid for the debt.
In which case the Crowdstrike shareholders should be suing Delta for devaluing the Crowdstrike stock?
If 99% of Deltas computers were disabled by a virus because they hadn't had AV updates pushed out to them during the canary phase, I expect they'd sue in that case as well.
Vultures circling vultures (with all due respect to vultures).
What gets me is the inferences that Delta's systems were down for a long period of time. Makes me wonder if that is true, and indeed,what kind of DR / snap shotting they have (unless it is all tin).
Maybe that inference is wrong and their problems are down to the sheer number of planes being in the wrong location - but I always that is a bit of an odd one. Surely they have replanning systems somewhere for this kind of scenario where they can "reset", tell the system where specific planes are and it can make best use of the planes to get the airline company running again (mostly)
Yes, it's hard to see how Crowdstrike wouldn't paint Delta's long recovery time — in contrast with that of most other victims — as evidence of Delta's own incompetence. I can see that playing well to a jury, if Crowdstrike go for a jury trial (though that'd be a gamble), and possibly to a judge as well.
I think Delta's chances are overrated by some commentators, and frankly hiring Bois looks to me like posturing, a projection of strength in the hope of getting Crowdstrike to settle and deflect the attention of Delta's own shareholders and board. "We're so sure it's not our fault that we hired an expensive lawyer!" The best defense, and all of that.
Delta has a pretty good chance of succeeding
Do they? I'm not so sure.
I think Crowdstrike have demonstrated remarkable incompetence in this case, but actually winning this sort of suit has historically been a long shot. Crowdstrike might decide it's cheaper to settle, but I'm not making any bets.
I wish whoever keeps downvoting this would explain what evidence there is for the probable success of a suit against Crowdstrike. All I've seen so far is a lot of unsubstantiated hope.
People are angry at Crowdstrike, and they had patently terrible practices. Neither of those are slam-dunks when this gets into a courtroom.
We don't know what was in the contract, and that is the important bit. We probably won't see that until discovery (and even then they'll probably not release it to the public). Big customers, remember, usually ask for specific contract terms that might now be turning round to bite CrowdStrike. Just because software is "generally" exempt doesn't mean that that general exemption necessarily applies in this specific instance.
They can spin up a Windows VM for testing, but the real cost would be hiring human QA testers to take responsibility for making the calls to delay a release. Other tech companies hire QA, but to maximize profit for the next financial quarter, you need to cut down on expenses like paying for employees. This is the strategy for optimizing for the short term over the long term.
I've said it before: to CxOs, I.T. departments should not even exist. After all, their vendors promised their services and products would be 100% trouble-free!
This is not even snark, just pure sad, sad truth. And even worse, I've been prevented form holding some of our vendors feet to the fire on the contracted SLAs! Which told me obvious kickbacks were now obvious. And other fraudulent shenanigans between vendor and my employer.
Can't they afford a test lab?
I am guessing they probably had an additional flaw in their process/CI/CD/whatever, such as not verifying that what they are pushing to the world is the same thing their QA approved. At least check the hash or something?
This is not instead of small yellow birds but in addition to them.
I don't think checking the hash would necessarily work. Create a crap file, calculate its hash and all the hash will subsequently confirm is that it's still the same crap file. It needs actual validation of the contents of the file, for instance, look to see if the memory it's about to access is legal and reject it if log an error message instead of going ahead.
Exactly. They've already admitted that "QA", such as it was, of channel-file updates involved running them through an analysis program which looks for certain types of errors — but not, as it happens, all of them. It did not (and still does not, as far as we know) include actually testing the system.
Crowdstrike have explained that they do not actually try to run their own software before pushing it out.
Existence of testing does not mean they have QA testers, because a type of partial testing called unit testing is done by developers. Developers write unit tests to test their code, which are typically incorporated into a stage of an automated DevOps pipeline that blocks the release if the unit tests fail. CrowdStrike's public statements indicate they don't have QA testers for the channel files, only unit testing.
See:
https://en.wikipedia.org/wiki/Unit_testing
They may have integration tests as well, which are written by the same developers who write the software and the unit tests.
One can't generally expect a developer to write adversarial tests against their own code. If they knew which corner cases to test for, then they would have written the original code to account for these cases instead of wasting time writing extra tests.
Unit tests are crucial when making changes to existing code, but they are useless in catching bugs. For that you need a QA team.
Unless you are apple the diversity of hardware, bios, OS and other configurations make a test lab difficult even if you can afford it trying to ensure that you have every possible configuration represented and the updates that will be applied by other vendors like Microsoft if you are CrowdStrike or OEMs like HP, Dell, Lenovo... if you are Microsoft have been tested with the patch you are putting out is a large undertaking, and a large part of the reason even when hardware vendors are onside getting full compatibility for new hardware in free open source software maintained by volunteers can take time. So whilst Microsoft have dumped a number of dud releases on us they have not quite messed up like this and being the vendor of a whole OS and security product for the same OS their record doesn't look so bad compared to the Crowd now.
Still given the over 660 million shareholders and a major fraction of the global population being customers, over 200 billion in revenue in 2023 one might expect Microsoft to keep improving.
Sorry, I disagree, you can't give Microsoft a pass like that, exactly BECAUSE they have the resources to do it right - all they do now is pass the burden to the customer.
First off, hardware diversity is something that you can address by layering the software. It's exactly because they sought to give their own software a competitive advantage that they broke that layering, so issues there are their own fault.
Secondly, their code is brutally insufficient, they got lazy through Intel using that to flog new, more powerful gear.
Thirdly, good testing would have avoided the lack of functionality, missing features and other issues that bedevil their 'new' version of Outlook and Teams - no hardware dependency there, it's good old UI and software design. Instead, customers were forced to waste massive amount of staff time yet again working around problem created by practically every new MS software release ever.
I've said it before and I'll say it again: if anyone was honest enough to also consider the OPEX expenditure associated with the use of Microsoft products they would not have gotten far.
In contrast to Microslop code, OS/2 is still as snappy as ever.
Sadly, it is more of a niche OS than a mainstream one.
Seems the higher the windows version, the more bloat...
Time to ditch the bloatedness for something more leaner and meaner...
off to ye pubbe to drown my sorrows ====>
Then DON'T try to defend Micro$hit. This is ultimately entirely THEIR fault.
They wrote an OS that allowed the ClownShit garbage to break booting at a low enough level that you needed a ButtLicker recovery key.
They COULD have locked out rootkits. They tried to blame EU rules, but all they had to do was remove their own products from having access at that low a level - which they obviously should have done.
They messed up, and messed up bad. It's not at all unreasonable to blame Micro$hit for bricking ALL of those computers.
They don't improve. They get worse.
It really is. They can't police every vendor and if they did you'd all be moaning about walled gardens and how you have to be free to do what you like with your own devices.
Turn on a fucking dime you lot.
I agree with the general thought, but Crowdstrike was only able to run at the kernel level because of a mandate by the EU. Admittedly, the kernel model is anticompetitive in that it only allows Microsoft created/approved services to run in ring 0 and does not have any other ring that can see all of the services that need protecting. Microsoft can and should be held responsible for that part of this issue.
No, the EU did not mandate kernel access, it stipulated merely that other vendors of antivirus software should have the same access as Microsoft's own products. There never was a need for kernel-level access but Microsoft was using it to give preferential treatment to its own products. Typical Microsoft and correctly slapped down.
Anti-malware doesn't need to run detection at kernel (or some higher-privileged) level, but ultimately response has to — at least the portion that blocks hostile actions. Otherwise privileged user-mode programs could bypass it.
And similarly it has to load as soon as possible at startup, or you have a race that could allow malware to bypass it then.
Running in an intermediate ring (if Windows provided one) doesn't help, because it would still be unsafe to continue if there were a critical failure in the intermediate ring.
Doing detector configuration in kernel mode is stupid, but that's on Crowdstrike. Nothing about Windows requires that. They could parse in user mode and then push valid rules down to the detector.
They might even be able to do detection in user mode using ETW, as others have suggested, though I'm not entirely convinced that's feasible and comprehensive, or how well it would perform.
Here's a tip: avoid the really-not-cute replacing of the 'S' in Microsoft with a dollar sign and in general the name calling and stick to the facts in your argument and you could have the chance of a good debate.
I know it's appealing to express yourself this way, but it creates two problems:
- you don't establish a good basis for a discussion, but worse ..
- .. this approach muddels your own thinking.
You effectively radicalise yourself by expressing yourself in this way, and that creates a mental bias that is not going to do you much good. Keep an open mind.
I too am absolutely no fan of Microsoft, but I also know there is absolutely no way I can steer people towards a better approach if I start off ranting at them with foam on my mouth - it will just make me look like an idiot. People who stay calm and avoid zealotry have far more impact (although I sometimes pretend to be one for the other side of a debate just to liven things up, but that's a conscious decision to either poke fun, or try out new arguments or generally be annoying :) ).
Now try again.
Ditto but in this case it wasn't Microsoft's update, it was Crowdstrike's, delivered by Crowdstrike's channel, not Microsoft's. It was Crowdstrike's responsibility to test before release and theirs alone and doubly so because it was applied automatically so it would be difficult for customers to test for themselves.
Hmm - ish.
On a tactical level it's Crowdstrike.
On a strategic level it's Microsoft because it's their bad security that (a) made things like Crowdstrike necessary and (b) allowed Crowdstrike so deep access that it could kill the whole machine.
Neither are without blame here IMHO, one for bad testing, the other for being the root cause.
If we let the root cause get away with it this is pretty much guaranteed to happen again.
(a) is poisoning the well. There are security issues with Windows, such as its large attack surface due to Microsoft dumping everything they think of into it, and legacy cruft like NTLM; but there are security issues with all OSes, and particularly with users. And Windows is a very large target. EDR would be necessary (under any reasonable threat model) for network-connected systems even if the Windows security situation were much better.
(b) is nonsense. Any kernel-mode driver can BSOD, and the response function of EDR, at least for function blocking, has to run at a level more privileged than anything in user mode, or it could be bypassed by privileged user-mode programs.
Microsoft potentially could have looked more closely at Crowdstrike's driver and told them that the architecture was unacceptable and rule parsing needed to happen in user space, and denied them WHDL approval until that happened. We can blame Microsoft for not putting more constraints on drivers in general; Linux does a somewhat better job of policing kernel-mode code, and Microsoft certainly has the resources to fund it.
But it's sophomoric to suggest that the problem here was "Windows security sucks" (true but irrelevant) or that "Microsoft shouldn't let Crowdstrike run in kernel mode" (false).
Known fact is that crowdstrike created a configuration file that made crowdstrike ‘s configuration file reader crash when it tried to read the file. Everything else was just a consequence of that crash.
Such a reader should be written and reviewed to make sure that it’s not going to crash, whatever it tries to read. That’s how it works everywhere else. And that can be done without looking at every possible input. Actually it must be done _without_ looking at snh specific inputs.
Crowdstrike might have figured out that they really need to have organized testing but its really too late. This fiasco calls their entire software engineering capability into question -- until proved otherwise what they appear to be vending is a bright idea that was sloppily implemented backed by a first class marketing and sales effort.
Sounds like weasel words with built in get outs from crowdstrike.
Validated: is it the right product? No reference to quality.
Tested: anything can tested but what tests? (and did it even pass?)
Certified: according to what?
Could pass all those checks and still be a buggy thing that brings down global IT.
There were/are configuration options that would have delayed the rollout and let the customers perform their own canary tests. Recommended or not, they were not exercised by the victims' IT teams. This is a good reason to TEST TEST TEST and stop firing the folks that would have prevented this from happening at your company! The vendor is not the one signing up for the risk here... it is the plaintiffs. I don't see lawsuits going much of anywhere. But maybe the point is only to bleed Crowd strike a little?
It seems that those things did not, in fact, exist.
It appears that it was only possible to delay engine changes.
It was a data update that trashed the place, and it is pretty clear that said update had never been loaded into a running system anywhere before it got deployed to the world.
Delta will probably win their lawsuit, the pension fund is quite likely to as well - they're both based on the same claims CrowdStrike made that have been proven false by events.
Neither of them will get much actual money though, because CrowdStrike will be bankrupt.
So the developer who created some new anti-malware code and modified the configuration file to enable his fix never used the configuration file reader to read that configuration file. There was a too supposed to test configuration files that was reportedly never run. And it is possible that the configuration file was 100% valid and it was just the reading software that crashed, so verifying the configuration file might not have found any problem.
As always, bring in the lawyers.
It was an accident. Shit happens. No matter the testing this stuff happens. I'm a developer for a large company. I've seen people make updates that worked fine in testing, but still broke the production environment. Testing environments are good and catch a lot of stuff, but they still simulate what actually happens in production. Sometimes stuff breaks. I hope they throw out all these lawsuits that are coming.
@BasicReality "I'm a developer for a large company. I've seen people make updates that worked fine in testing, but still broke the production environment."
Only because the test plan failed to cover that production scenario.
"Testing environments are good and catch a lot of stuff, but they still simulate what actually happens in production."
Meaning you've "seen people make updates that worked fine in unit testing" and then deploy the updates to production without system testing the updates. What CrowdStrike did is no different, they deployed the update to channel without any system testing.
If CrowdStrike had spun up some Win vms running Falcon before the channel went live that would test the upgrade on the system. Even a single vm would have blue screened when getting the update. Instead of 8.5 million machines.
"Sometimes stuff breaks."
And testing prevents stuff breaking being released.
Shit doesn't "happen". There's always a cause though it may be hard to peel away the layers.
There's a large industry of folk who chase down events. My current fave is the Buncefield incident (but only 'cos I was involved in a small way with some of the steps to mitigate against it happening again.)
The lawsuit is for material misrepresentation. CrowdStrike's matching update process to their SEC reporting will require extreme pretzel logic to avoid a finding that they lied about how they do business. This matters because these statements are used by investors to determine the risk and future growth of a business.
1) Regarding this shareholder lawsuit, I don't expect it to actually get very far. Why? "There is no such thing as bad press." Given that my company tapped the brakes for about three days before going ahead to implementation, I expect their stock to be UP in three months BECAUSE of this incident.
2) When you're talking about millions & millions of servers, you don't to canaries, you do rings. Each ring as ~5x as many servers as the one before. For your larger customers, you force some of their systems into the earlier rings. Basically, customers set the order for the rollout.
3) Their password requirements contradict NIST. And they don't tell you, but "+" is not allowed. Which means that they are storing the password somewhere.
Deep, deep cultural failures here.
Wait, WHAT?
Your company didn't have ClownShit before, ClownShit fucked up millions of computers, and NOW your company is installing ClownShit software?
Find a new job. That's some serious industrial grade stupidity going on there, if they'll do something THAT stupid, they'll be going out of business soon. Get something else on your resume before it's tainted with whatever soon to be failed company you work for.
So much wishful thinking around the Crowdstrike disaster. Have any of you people paid attention to IT history?
Companies fuck up hugely all the time. More often than not they get away with it. Sometimes they're rewarded.
Being angry doesn't make you omniscient, and the IT industry is neither meritocratic nor just.
Boilerplate response. Of course you do. Unfortunately for you, just about everybody else probably thinks is has merit, and I'm guessing the judge will too.
That said, I think there should be a law stopping shareholders from sueing their own company. You're not happy with the company's performance ? Then sell your shares, take the loss and go invest in a better company.
Companies are not permitted to deceive their (potential) shareholders. That's actual law, and an absolute necessity for a functioning market.
However, you've only got standing if you took a loss because of a deception.
So if shareholders cannot sue, then companies can lie with impunity - and the entire stock market collapses within about 39 seconds.
"Company" is a collective word for people. Who are the people who comprise the company?
The directors shouldn't deceive the shareholders. The manglement shouldn't deceive the shareholders. It makes no sense to say the shareholders shouldn't deceive themselves at least not collectively and in the legal sense.
The underlying concept of the stock market is fine; it allows people to invest in large companies (too big to exist as sole trader/partnerships) with limited liability (the most you can lose is your investment) and lets the shareholder trade their investments. The issue is how it's been abused over the years by people extracting money from it, between shorting stocks, microsecond transactions, etc, etc.
In terms of "not lying", most of the theory of capitalism and economics relies on "perfect information", which is why in practice it's horribly broken, but restricting the lies is intended to help.
There is a vast gulf between "Crowdstrike sucks" and "Crowdstrike is legally liable".
As others have pointed out, this case might (might) be viable on misrepresentation grounds — but holding a company liable for misrepresentation based on vague statements about quality and fitness for purpose is very difficult, at least in the US. Did Crowdstrike claim to investors that they perform system tests before pushing channel updates? Without a specific untrue claim, it will be difficult, particularly for investors, to prove misrepresentation.
Note that there are no regulatory standards (outside certain narrow contexts) for things like software testing, so plaintiffs can't claim that they could reasonably infer particular practices on Crowdstrike's part. (Well, they could claim it, but they don't have any grounds to support it.)
I'd love to see more precedent for liability in the software industry, but I'm not holding my breath.
@Michael Wojcik "As others have pointed out, this case might (might) be viable on misrepresentation grounds"
The filed lawsuit is 29 pages long, but the misrepresentation claim is section 3 on page 3.
"3. Throughout the Class Period, Defendants (defined herein) repeatedly touted the efficacy of the Falcon platform while assuring investors that CrowdStrike’s technology was “validated, tested, and certified.” This complaint alleges that these statements were false and misleading because Defendants had failed to disclose that: (1) CrowdStrike had instituted deficient controls in its procedure for updating Falcon and was not properly testing updates to Falcon before rolling them out to customers; (2) this inadequate software testing created a substantial risk that an update to Falcon could cause major outages for a significant number of the Company’s customers; and (3) such outages could pose, and in fact ultimately created, substantial reputational harm and legal risk to CrowdStrike. As a result of these materially false and misleading statements and omissions, CrowdStrike stock traded at artificially high prices during the Class Period"
In spite of CrowdStrike’s anti-threat update procedure the statements “validated, tested, and certified.” are still true. Also (3) such outages could pose,... was covered in the SEC filing quoted in the lawsuit. Also had CrowdStrike disclosed their anti-threat update procedure would CrowdStrike stock have traded at a lower value? Probably not.
Finally “validated, tested, and certified.” is only mention twice in the lawsuit. The second mention is the only example of the statement being used, which was in an earnings call. For such an important point to the claim I would have expected more than a single example in the lawsuit..
They don't believe that, no-one believes that they believe that, everyone- them and us- are in in the fact that PR will say that regardless if it's in their interests to do so, or that of those who- having fucked up- have to at least go through the motions of pretending that the case is defensible for their own sake.
They know that we know that they know that it's bullshit, the game is played regardless.
Of course this is a boilerplate response from the legal team and no one pays it any attention (except Internet busybodies); but even taken at face value it's correct. They're saying the case has no merit, and that's a reasonable position to take, because it's very difficult to establish liability or misrepresentation in the software industry.
They didn't say Crowdstrike weren't at fault for their screwup. They're talking only about the case at hand.
Bottom line is crowdstrike is a good peice of software and well supported, but....
They made one stupid mistake (hubris)... and for that they should die
like it or lump it..... Capitalist system is cruel and aggressive
the underlying tech will survive on its own merits. it is the best of the bunch
unless microsoft aquire it and integrate it into there absolutely shite joke competitive system
because no matter how high quality and clean anything is, put it in a bowl of shite it will never come out clean :)
Bottom line is crowdstrike [sic] is a good peice [sic] of software
No, it really isn't. Parsing configuration in kernel mode is a stupid, lazy architectural decision. The parser is brittle, and error conditions are not checked for or handled adequately.
The testing regime is patently terrible — and we have Crowdstrike's own description to base that on. The distribution regime is terrible, and configuration-file distribution ignores the package's own phased-deployment options.
I'll also note, anecdotally, that I've been working for a Crowdstrike customer (thus the anon post) for a few months, and I've seen a couple of dozen tickets raised with the helpdesk because of huge performance issues or other adverse effects.
There is no doubt that CrowdStrike has a mess on their hands and that mess is going to be ugly and expensive. But, before we completely crucify them maybe we need to look at some history and how we got here. Channel files exist for a reason, they are updated the way they are for a reason and CrowdStrike got access to write Kernel drivers for a reason. That reason is market demand...Go back in time and look at how some of the antivirus definition files had to be distributed via SMS or some kind of script. The unacceptability of such a time delay between a known signature and an AV update was too great and lead to some of the early virus variants being able to spread like wildfire. The solution..... "Channel Files" they update automagiclly, they meet the need for a quick reaction time that security is demanding, there is not testing delay from the customer because most don't even know they have an option to say no to them and lastly they come from big trusted names in the industry. Certainly they are doing testing before they push them out (sarcasm added for affect). And for years this model worked great, until it didn't. Some readers will be old enough to remember when Symantec did nearly the same thing back in the day and broke a bunch o shit. We the customers are asking nearly impossible things from CrowdStrike. React instantly, never miss a threat and never make a mistake. I get that we pay them well for what we are asking BUT I think back to my first project management training almost 39 years ago when the instructor explained the Triple Constraints of Time, Quality, Money and that you can only have 2 out of the 3 at any given time. In order to get the Time line we are demanding at a price point we are willing to pay CS had to make choices, one of the choices we ALL agreed upon when we allowed them to update Channel files without our own internal testing or even change control approval was that Quality might suffer. From all I have read, this was a human error. A big one and an important one but a human error none the less. If we want perfect we are going to either have to sacrifice speed or money. That's the unfortunate reality of the tripple constraints. No matter how much the Board or the C-Suite does not want to hear this message. There are some facts that even the all great and powerful cannot change. Just my thoughts, feel free to disagree.
This “canary” request is only one of several items, and not the most important one by far.
1. Crowdstrike had and probably still has software reading configuration files that crashes when it is given data that it doesn’t like. With adequate code reviews that should never have happened.
2. A developer created code meant to fix some problem. That developer should have true what happened with his changes and the modified configuration file, which would have revealed the crash. What did the developer actually test?
3. A script was supposed to be run before deployment that checked the configuration files. This didn’t happen. It is also possible that the configuration files were correct and verifying them wouldn’t have found any problems. So the check should have included actually reading all the configuration files.
4. If an update like this one got through, not crashing would have been so important, there should be an automatic fallback to the last known working version.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
