Do you really need that GPU or NPU for your AI apps?

There's no avoiding AI and LLMs this year. The technology is being stuffed into everything, from office software to phone apps. Nvidia, Qualcomm, and others are happy to push the notion that this machine-learning work must be performed on an accelerator, be it a GPU or an NPU. Arm on Wednesday made the case that its CPU cores …

[ecofeco]

Need what for what?

I won't need something for something I won't have.

Problem solved!

[ThatOne]

Re: Need what for what?

Come on, they will keep repeating that AI is inevitable, up there with taxes and death, until people start believing it and stop wondering why they should ever need this (enough to spend a fortune just to accommodate their imaginary "AI" needs). The technical term is brainwashing.

_{I prefer to keep my dirty mind.}

[Anonymous Coward]

"There's no avoiding AI"

Farcebook recently let me know that by the 26th of this month they'll start feeding my data into their LLM, regardless of my objections to AI, even if they accept them.

See 'Their Terms' here: https://www.facebook.com/help/contact/6359191084165019

I've downloaded my data already and will move to Mastodon shortly. It wont be federated and I'll invite family only.

I've managed to convert both my aged father and brother to switch from Windows too, so they can shove their forced AI shite up their asses too.

[UnknownUnknown]

That’s a GDPR/CCPA violation, not a Term and Condition.

At a glance .. GDPR sets out seven key principles:

Lawfulness, fairness and transparency

Purpose limitation

Data minimisation

Accuracy

Storage limitation

Integrity and confidentiality (security)

Accountability

[HuBo]

Sprechen sie kleididoscope?

Depends on the AIapp I guess. Ever since the most-cited 10-minute HowTo we've known that large autocomplete cataplasms can run like bats straight out of hell on our old-hardware laptops, in our mom's basements (Core i3, integrated "GPU"), sweating only few bullets. Renesas and PlumerAI meanwhile suggest People Detection AI Vision can run on a €10 microcontroller (RA8D1, Cortex-M85) thanks to the ARM's MVE Helium uplift ... and without the giggles of Nitrous Oxide! Heck, even the Matrix-inspired continuous spinal-tap mind-slurp that is MS' Total Recall's 6th-Day syncorder doesn't seem to need a Copilot++ NPU to execute its docility virus spreading subroutines, so ...

... that leaves only Palantir's Maven Smart System of automated dissent-dissolution IMHO. My Sunday tribute to Tobias' right-on 17:20 pre-closing statement on this then, is this quiet interlude of relaxing musical nostalgia.

[jlturriff]

Consider providing transcripts of Kettle sessions

Some of us aren't easily able to handle video blogs. Can we get transcripts so that we can know what happened?

[BenMyers]

Excellent insights!

Thank you for dealing with at least some of the AI hype.

[beeka]

The answer to the question about "when is something AI vs engineering" is that all the while you understand exactly what it is doing it is called engineering. We don't understand what AI / ML is doing so it can't be trusted with anything important. Small ML tasks can be understood well enough, e.g. a neural net for character OCR, but usually as part of a larger, well-engineered, system.

That it is possible to feed billions of linked tokens into a computer and it can guess what you expect it to say next feels like magic: I could just as well be speaking with a Sophon. The Raspberry Pi foundation encourage the use of the term Machine Learning in an attempt to increase the understanding of the limitations and reduce the mystique. I've been trying to understand these systems for the last few months and am still not sure what a GGML file contains and whether the differences in models are due to embedded run-time logic or alternate training data / processes.