This image-recognition neural net can be trained from 1.2 million pictures in the time it takes to make a cup o' tea The shortest time for training a neural network using the popular ImageNet dataset has been slashed again, it is claimed, from the previously held record of four minutes to just one and a half. Training is arguably the most important and tedious part of deep learning. A small mountain of data to teach a neural network to …
Of course not. You make ~23 cups of tea per second, everybody knows that!
I was going to suggest using a pressure cooker to raise the temperature of the water above 100 Deg C and thus shorten the tea brew time, but as some clever soul on Reddit has noted it takes pressure cookers a few minutes to get up to pressure.
https://www.reddit.com/r/tea/comments/27pcx9/tea_under_pressure/
That ~23 cups per second claim is rather dubious...
According to their data, they have 140 litres of water on board and it uses 1000 litres in 25 minutes. Therefore it has a runtime of ((140/1000)x25) = 3.5 minutes = 210 seconds.
210x23 = 4830 cups of tea per tankful.
140 litres (remember, the tank capacity) / 4830 cups = 29ml per cup. That's slightly less than 2 tablespoons per cup. If someone served me that as a cup of tea, I'd be mightily disappointed.
A Nice Cup of Tea by George Orwell
Evening Standard, 12 January 1946.
The subject is a serious question. Will you get a comparable outcome, say, with 120K images and 10 GPUs? With 12K images and 1 GPU?
On a less serious note, I can't get rid of the following line of thought easily: what is the fastest way to get "trained" to a level of an academic degree?
1. Pay up front to a "correspondence university" in Eastern Europe or South-East Asia or elsewhere.
2. They will declare you "trained" really, really fast, maybe even faster than you can brew a cuppa.
3. It will all be rather artificial, you won't get any real knowledge or skills, but answering questions like "is it a cat?" with 58% accuracy may still be a realistic outcome. Actually, 58% of your subsequent decisions in a managerial capacity may be correct, too.
And back to the serious mood again: it is easy to mock such a "record" given the demonstrated lack of test accuracy, but a negative result is just as important as a positive one, assuming it is novel. It may tell people, don't go there, and it may point someone in a direction of improvement.
58% accuracy is little better than chance, if you're in a strictly limited domain of potential answers (i.e. it's not able to suddenly pop up and go "Well, that's obviously a cello-playing giraffe balancing on a chair surrounded by a Monet-style backdrop with a greyscale filter in the upper-left quadrant", but has to answer "giraffe").
Depending on how the test is set up, you could do better with a dice-rolling robot. The problem is that it's not making decisions. None of these things are capable of inference. It's adding up and then giving the highest-number. They are trained on data to give a limited set of answers and that's it. They are the production-line-workers of the AI world, but literally only ever capable of doing the mindless "put the cap on the bottle" job - and not even approaching the capacity that even the dumbest of humans/animals has inherently.
The problem you allude to is that it's not how fast it's trained but the results that it gets. In the same way that someone going "I got my degree by writing off to India" isn't at all impressive and probably wouldn't ever be employed on that basis.
And... sorry... but just look at the power required to train that model to get little-better-than-chance in a very limited-domain problem. And we have *zero* control over that model. To "untrain" it would take years of processing, not days. To "retrain" it to another purpose (or to refine its existing training) would be the same job. Any kind of machine learning plateaus REALLY quickly. And we have no idea what it's actually basing its decisions on or how to modify them.
Machine learning really is the worst kind of unscientific black-box to employ in any task of even the smallest importance.
The meaning of "accuracy" is important in this context. The ImageNet database consists of images that have been categorised using the WordNet tree structure. So, for example, part of the tree is Sport / Racing under which you'll find Greyhound Racing, Horse Racing, Boat Racing and Car Racing.
Accuracy in the ImageNet challenge appears to involve putting the images into the correct category as originally labelled. I'm not convinced that's a terribly useful definition of accuracy. For a number of practical purposes, you might be able to live with an image being wrongly categorised as "horse racing" if it was actually of a dog track, because it's in the correct branch of the tree and of a similar nature. On the other hand it's unlikely that there are many circumstances in which miscategorising it as "boat racing" would be even partially helpful.
That's the trouble with ML, at least in my uninformed opinion - there's a lot of focus on solving artificial problems that dpn't necessarily map to real world appilcations.
I did discover one bit of ImageNet that might be a particular challenge: there's part of the tree that's Animal / Male / Horse under which (both physically and taxonomically) you'll find (or not find) "Stallion, entire" and "Gelding". I couldn't find "Brass Monkey" in the dataset, though, anywhere: perhaps the accuracy would depend on the effectiveness of GPU cooling.
It's the AI winter all over again, just with improved technology. After genetic algorithms was now the in thing, we now have neural nets and deep learning. Don't get me wrong, the stuff we are doing now is phenominal and a step change from past attempts. But as you rightly point out, a computer would remain utterly unable to comprehend meaningfully something as simple as an image classification task if not trained extensively. Training it to handle all aspects of life and become a "general AI"? Not a how. Magnitude of computing power has helped remarkably, but bear in mind the human brain does this on 20 watts. We have much to learn. fMRI and other techniques have given us just a fraction more of a glimpse at how our remarkable brain has eveolved. I'm sure more practical and fundamental research will advance this area rapidly.
I share some of your commenters' concerns, such as how can we reason about an AI system? This is being addressed, but, frankly, the problem runs much deeper. We are not (yet) able to effectivly articulate how a trained neural network works. So now the big deal is the ability to described "how" a neural net arrived at a certain result. But before long, works like "how" start to lose their mening. "How" did I decide to get up and have a sandwich today? "How" did I choose my academic and professional career?
As I said...much to learn.
But, what a time to be alive!
I think the main problem is when faced with a choice like
- this is a $x
- this is a $y
there should be an output option of
- I have don't know what this is
Children ask 'what is this thing' and any real learning system needs to ask if the answer is less than 90% sure.
... for those of you who want an ElReg mug (and you know you're out there!), they used to be available from the Cash&Carrion store. Unfortunately, this went TITSUP[0] back in 2008 ... There was a wild rumor that they were bringing it back in 2014, but nothing ever came of it. Now, I don't work for ElReg and certainly don't speak for them, but I rather suspect that if enough of all y'all were to ask for such merchandise, it might reappear. Or not.
Personally, my copy of this shirt is getting rather threadbare, but still getting a lot of reactions whenever I wear it. I'd purchase another in a heartbeat if they were made available. And another two dozen assorted vulture themed mugs would look good at the guest coffee service.
Over to you, commentards! As we all know, the squeaky wheel gets greased ...
[0] Totally Incapable of Transferring Selected User Packages
I'd rather click a button and give El Reg a fiver directly than buy a basic branded product for a fortune and giving most of the cost to Cafepress or similar.
But websites never bother with that option. Instead it's *HUGE* sidebars adverts and "pay £20 to give us a pound and get yourself a t-shirt that won't last a handful of years".
"
... albeit it with a 58.2 per cent accuracy.
"
So a bit better than flipping a coin, though not much more. And this type of kit is what will be used to decide whether or not to arrest and detain random people in a crowd monitored by "AI" cameras. Surely cheaper and no less accurate to simply arrest everyone wearing a blue shirt.
"So a bit better than flipping a coin"
Assuming they used the highest level categories there are around 30 so random chance would be more like 3 - 4%, if they used the lower level categories (somethng like 20k I believe) then you're looking at a fraction of a percent...
However no point in doing in that fast really when you can train for a bit longer and get to ~90%
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
