How to stealthily poison neural network chips in the supply chain Computer boffins have devised a potential hardware-based Trojan attack on neural network models that could be used to alter system output without detection. Adversarial attacks on neural networks and related deep learning systems have received considerable attention in recent years due to the growing use of AI-oriented systems …
... any chip design I've been involved with has used _all_ the available silicon area -- meaning that there is little room for widespread switchable functionality.
And why is it necessary to have switchable functionality for this attack to work? Well, I presume there is some automated acceptance testing to check for functional/non-functional chips. And, if you're going to test the chip, it might as well test whether umbrellas can be distinguished from Bolivian Seaborne Marching Powder.
I believe the usual figure cited for getting someone to spill commercial secrets belonging to his employer is on the order of five times salary.(1) Offer someone five times his normal salary, and most people will spill the beans. I'd guess that sabotaging your company's products is probably similar in price.
(1) There are a few people who won't at all, or who will demand substantially more, or who will take it as an invitation to be a double agent, i.e. take it immediately to the boss: "Company X offered me Y dollareuropounds to spill secrets. How can we creatively misinform them, ((and what bonus will you offer me to do so))?"
.....the difference between neural networks implemented in hardware (this article's subject), and neural networks implemented in software?
*
I'd be more worried about attacks on software implementations, not least because the software can be modified by the attacker......where I'm assuming that a hardware attack, once found, can no longer be used by the attacker.
That was a measured counter-point.
I'll be a little less measured.
There are very few introspection tools to understand how NNs are actually classifying, and those that exist are mostly for images. We can visually understand "oh, so it's recognizing a round thing in that layer, and calling it a wheel over here", but not well enough to change the NN in a way that still works.
We are very far away from a small automated system that could subtly change a trained NN to do something specific. To put it into genetic manipulation terms, it's quite easy to kill something. You need to be very sophisticated to make a fish glow. You must be vastly more sophisticated to change a chicken into a fire-breathing dragon.
I read somewhere that neural networks are susceptible to the same sort of problems that humans suffer when viewing images where they "look like" something familiar.
Case in point, some university showed a previously trained neural net pictures of clouds (obvously random) and the network classified them as pictures of: animals, plants, chairs etc.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
