It's all relative
Seeing as the entire marketplace is full of insecure products, in this case "best" probably just means "least bad".
Which? Magazine has been called out for recommending a line of smart home cameras with known vulnerabilities. The Consumers' Association magazine has worked hard to build trust in its consumer-focused product reviews. The fact that the Samsung SmartCam SNH-P-6410 smart home security camera still has Which's "Best Buy" …
There is likely one secure solution out there, in the cheaper price range.
It is called Raspberry Pi ZeroW + camera.
Closed source solutions are crap, I have them, but on independent VLAN, and with no Internet access.
We need more open source cams. Most chinese cams are running Linux anyway, so it would be trivial for the vendors to publish specs on the DSP and camera hardware, and thus let us create open firmware.
The first to do it would get lots of business, but likely would suffer from customers not upgrading their crap as fast.
Sadly, this is common place with Which. I don't think I've seen them once take security into consideration with any of their evaluations or reviews of tech gear.
I don't want them to go full tinfoil hat, but it would be nice if they at least mentioned the downside to their members plugging their private life into a tech company's ecosystem. But each new potential data slurp and info leak gets an uncritical thumbs up.
It leaves me wondering what I'm missing in the other consumer goods they cover, but where I lack any expertise.
I gave up on Which? a long time ago (end of the 90s). I used to subscribe, but they started reporting on IT related products (and automotive) and there were such huge, glaring inaccuracies in what they were detailing, that I felt I couldn't trust them on anything.
And that was before we even got to the security aspects of products.
I felt, if I couldn't trust them for products that I know something about, how can I trust them on subjects I know nothing about?
I gave up on Which? a long time ago ... I felt, if I couldn't trust them for products that I know something about, how can I trust them on subjects I know nothing about?
Ditto. I recall a review of either ISPs or email providers and stated that spam wasn't a problem - when the only reason they didn't get spam was starting with a fresh address and making conclusions after just a week !
But yeah, in several areas where I had some knowledge, I couldn't help but call "bulls*t" to some of their statements.
Another Ditto? I found that if I read the Which forums, they were slating "Best Buy" products and recommending others. I got a very nice vacuum cleaner that way. But that taught me to take Which advice with a very large fist full of salt. I didn't even bother looking at the IT reviews to be honest.
"I gave up on Which? a long time ago (end of the 90s)."
I gave up before then when they recommended as a learner motorbike a Kawasaki with brakes totally inadequate for the speeds it could reach and with somewhat limited turning abilities. It could at least have reduced the number of motorcyclists on the road, but I don't think that was the intention.
From the Which website; "Which? test labs
Every product we test and score at Which? goes to our independent test labs to put be through its paces by qualified and experienced experts. This ensures that everything is tested fairly, impartially and consistently, so that we can continually offer the best advice to consumers."
It looks as though they need someone to test their testers.
Personally I have never been that impressed with a lot of their recommendations.
Indeed. Long, long ago I used to read Which? reports with lots of interest as a great source of information. Then I read one or two reports into subjects where I had some expertise, and saw a different side.
Basically, a lot of what's there is "how happy are the owners with a product"? That leaves a situation where owners of a cheap product take the view "yeah, it's fine, does the job, I'm satisfied", whereas those who take a serious interest in a subject and buy top-end gear remain sensitive to its flaws.
The importance attached to security would seem still to be something that depends heavily on ones perspective, so IT practitioners differ radically from Joe Public. Some journos are working on that divide, but I guess they still have a way to go.
Has anyone (here) studied the actual vulnerabilities under discussion, and where they fall on a scale of hypothetical to easily exploitable by a stranger?
Was established by Sir Mortimer Lefancy in 1867, based on approved journalistic reviews of the time. We make it evolve continuously, once per geological epoch.
Come on guys, if you can't be bothered to Google, don't go pulling a "we value our customer's privacy" shit.
You didn't, and you don't have a clue.
End of.
Have always been poor.
They are probably fine for reviews of (non IoT) fridges, washing machines etc. but anything with a decent "IT" component then reviews are worth treating with a pinch of salt (by pinch, I mean an amount worthy of daily production of a salt mine)
Which don't seem to pay much attention to their members feedback either. I've had problems with both an induction hob and an electric blanket that I bought because they were recommended. Afterwards, in each case, I found lots of member comments who had encountered the same problem. But the which recommendations still stood, for years in the case of the hob.
So are there any good security cameras or reviews of them?
"So are there any good security cameras"
There are plenty, but none of them are ones that connect directly to the internet. Honestly, though, that's what you want. From a security point of view, better to have your cameras talking directly to a computer on your premises, which does the video storage, etc. You can then use software on your computer to make the video stream available over the internet if that's something you really want.
"Which? found a minor privacy concern with this device..."
Where on earth did they find a "minor privacy concern"? All the flaws reported here were pretty much worst-case vulns (total stream takeover.) The only thing worse would be rooting the device. That is major, not minor.
So did they discover some more vulns, or did their spellchecker replace "major" with "minor"?
Everyone else has already said it above. When Which? talk on an area that you know about yourself you then realise they are not always that on the ball. In the IT world especially.
What I find comical is that they use their subscribers to get most of this feedback. So they are asking people with experience of just the one item they bought and not someone who has experience of the market. Even a 20mins phone call with people like the Pen Test people would have lifted the quality of their reviews.
I know how bad their advice can be because they have asked *me* to provide opinions for them! I got my 30 seconds of TV fame by doing an interview on their behalf.
It is not just the dodgy webcams and IoT devices. You can see it in many other reviews. Read between the lines and you can see too much personal opinion in there.
They are great on some of their campaigns, and know how to run White Goods tests into the ground, but they need to know when they are outside of their own knowledge area.
Actually - that's a good point. I have been meaning to cancel my subscription for ages. This is a good nudge to go do that.
- and not just "IT" ones.
Years ago, I worked for <redacted> hifi manufacturer. We had two brands, cheap and reassuringly expensive. Each brand had a model of bookshelf speaker. Which included them in a "group test".
The expensive ones came top. the cheap ones bottom.
The products were internally identical, the only difference was case finish and trim.
Mind you, Which? was *still* better than the typical hifi mag review.
Ah, but you presumably haven't taken into account the improved hi-frequency fidelity amplification designed into the superior speaker case trim, and the significant improvement to the lower registers from the more expensive case finish. It can make all the difference between exceptional sound and merely very good.
Which? bases best buys on a bag of criteria. The testing is apparently reliable on a criterion by criterion basis ( though IT doesn't sound great). But the weighting of these, things like ease of use, reliability, efficiency etc. is inevitably a subjective choice. You're going to end up with a judgement that trades security against lens quality. And so forth.
Which? bases best buys on a bag of criteria.
Having a pseudo auditable assessment criteria doesn't make the outcome any good. This month they're recommending as a "best buy" a £990 home coffee maker. And even then it only got 76% across their weighted criteria. Likewise, it busies itself reviewing hundred quid toasters, five hundred quid vacuum cleaners, and so on. Car reviews have been getting progressively more ambitious, including those popular-with-Which-rank-and-file models such as the Porsche Panamera, the Mercedes CLS, and the Tesla Model S.
When it comes to (say) energy, broadband or insurance, Which treats its readers like simpletons without the skills or confidence to make any decisions for themselves, but then goes into some reasonable detail on pensions, will and later life care. With product reviews increasingly for expensive products, I conclude that Which has degenerated to a general interest magazine for wealthy pensioners
I seem to recall posting about this before, after looking at Which? reviews of handheld vacs
Two products had similar performance and ratings.
The Dyson was "good value" at £100, a "lightweight" 2.1 kg and ran for an "amazing" 18 minutes
The Vax was "pricey" at £60, a "hefty" 2.0 kg and "barely" lasted 20 minutes
But at least they usually try to compare like-for-like, unlike most El Reg reviews ;-)
You're going to end up with a judgement that trades security against lens quality. And so forth.
So, we're going to end up with a judgement of a security system being superior because it has an on/off switch accessible from the street.
Got ya!
I'll stick with my original assessment the first time I read their drivel, buy anything not listed by them. They're as bad as Consumer Reports, who reported that RCA VCR's were superior to Hitachi VCR's, despite the fact that Hitachi made them for RCA and component for component, were identical.
Which should give an indication on my views and experience with bullshitting rating disservices and sites.
...I've just had an email from Which asking me to do a survey. One question was: "If you could pick one consumer issue you would like to see Which? campaign on, what would that be?"
I replied with: "Improve the accuracy of articles on your own website. See: https://www.theregister.co.uk/2018/10/08/smart_camera_which_wtf/"
I find the Which? reviews useful for seeing what's on the market and what all the bells and whistles do. They seem OK on relatively straightforward things (e.g. the toaster will burn the toast if you use it again immediately) but they are often well out of their depth even on slightly technical thingies.
Not so long ago I was amazed at their review of DAB radios which completely failed to mention the need to check for the Digital Tick. Many well known retailers such as Tesco and John Lewis are still ripping off their customers by selling digital radios that can't receive the ever-increasing number of DAB+ transmissions in the UK, but Which? readers would be none the wiser.
The Consumers' Association magazine has worked hard to build trust in its consumer-focused product reviews.
The Consumers' Association magazine has worked hard to market itself in the same way as Readers' Digest, Automobile Association (in their heyday) and all the other outfits whose main route-to-market is direct mail. The quality of their product is concomitant with that approach.
FTFY.
A so-called consumer champion selling its product via a "free trial" and reliance on inertia not to cancel is seriously unethical.
This post has been deleted by its author
Thanks for that PLT stuff Which?, from everyone who enjoys the hobby of amateur radio. You seemed unaware of the potential downsides but it had to be better because it meant not having to run cables.
When I was a student, my Landlord subscribed to Which and I got to read it. I was never impressed. I didn't get the impression that the people writing the reviews had really taken the time to get to understand the products they were talking about. They seemed to be prone to getting dazzled by looks and glossy-brochure claims.
I don't claim to be an expert on the things they were reviewing but, as an engineer, what I read didn't look like the words of people who gave sound evidence-based advice.