How Google hopes to build more efficient, multi-capability AI systems

Google says it is developing an AI architecture that can be used to train one giant system capable of performing multiple different tasks more efficiently than what's possible with today's models. Machine-learning models are typically built to tackle a particular challenge, such as object detection or facial recognition, and …

[RobLang]

Ensemble neural networks are still separate models

While the architecture of Pathways is extraordinary and to be applauded, I think there is a leap taken here:

> Pathways will enable a single AI system to generalize across thousands or millions of tasks, to understand different types of data, and to do so with remarkable efficiency – advancing us from the era of single-purpose models that merely recognize patterns to one in which more general-purpose intelligent systems reflect a deeper understanding of our world and can adapt to new needs

That's not quite true. In ensemble neural networks, you have separate models each trained to be an expert in a single thing. You have a language model, an image recognition model, etc, each are called an "expert" and thus an ensemble of experts. When an input is presented to the trained network, it picks the expert that is most appropriate and shows the input to that one. That network then performs its analysis and gives an output.

From the outside, it's sort of general purpose because you only need one system running to do different things but it's not really general purpose from an AI perspective because inferred knowledge is trapped in each of the expert silos. You can tell that this is the case because truly general purpose AI has the problem of encoding inputs from different domains - how do you represent an image in numbers in the same way that text is so that knowledge can be inferred? That's a *very* hard problem that this paper doesn't address, so I'd draw the line at calling it general purpose.

For current deep learning, this is very cool indeed.

[steelpillow]

Re: Ensemble neural networks are still separate models

As I read it, Pathways is specifically designed to take things a step further, breaking down specific complex tasks into what one might call a layered sequence of simple tasks. Each simple task is then carried out by a specific expert and the experts linked to complete the overall task. A new complex task just needs the experts re-linking and the odd missing expert trained up. Pathways is used to manage all that.

[Pascal Monett]

"train separate models for each type of task"

Yes. Because whatever marketing says, it is not AI.

[steelpillow]

Re: "train separate models for each type of task"

Define AI. Then get others to agree with you.

[spold]

Multi-capability

Tells jokes, makes tea, takes the dog for a walk.