Re: Missing information - biased test method of Finite State?
@Jou (Mxyzptlk): "If the firmware images go back to about 14 years this rises a question how Finite State chooses which firmware image to test"
[Disclaimer: I have no axe to grind, I have no incentive to protect Huawei or Finite State, I have not verified the methodology properly. I have skimmed the actual report, not only El Reg's summary.]
Finite State do say they focused on the most recent firmware versions of the devices they analysed. Methodologically, I do not see statistics on, say, how much better or worse the recent firmware versions fare than 14 year old ones - and maybe Finite State could do such an analysis. There are, however, examples of pairwise comparisons (that look appropriate) of the most recent vs. older firmware for a particular Huawei device. There is also an example of a comparison between Huawei, Arista, and Juniper switches (that should be relevant for the "and Cisco aren't any better" argument).
Finite State conclude that (in the example study at least) the most recent Huawei firmware is actually worse than the older version, while Juniper are better than Arista who are better than Huawei (and there are whole categories of security problems where Huawei score very poorly while Juniper and Arista look very well indeed). I am a bit baffled by Finite State's way of presenting the results (they use percentages compared to the worst cases they have ever seen), but I suppose it may be OK for relative assessments.
Another big point is the finding that Huawei apparently use outdated components and outdated SW versions known to be vulnerable. It is not clear to me how that affects the "past 14 years" detail that seems to generate a knee-jerk reaction in this forum. I am not making any argument, mind you, except pointing out that "14 years ago is irrelevant because things have changed" is an assumption, not a fact, and the report seems to indicate big flaws in that assumption.
In any case, I have not done any serious analysis myself, but I wouldn't be too quick in dismissing the report based on a mention of "going back 14 years" in a summary article.
Biting the hand that feeds IT
