GPU goliaths are devouring supercomputing – and legacy storage can't feed the beast

The supercomputing landscape is fracturing. What once was a relatively unified world of massive multi-processor x86 systems has splintered into competing architectures, each racing to serve radically different masters: traditional academic workloads, extreme-scale physics simulations, and the voracious appetite of AI training …

[Anonymous Coward]

There are still many scientific computing challenges for which GPUs are completely irrelevant. Vice versa, there are few actual useful applications of A"I" so the stampede towards GPUs is ill advised. But, tulips.

[village-id10t]

few applications

"There are still many scientific computing challenges for which GPUs are completely irrelevant"

Name one. If I can sort through a giant dataset in a fraction of the time using my GPU that's not irrelevant.

"there are few actual useful applications of AI"

Pretty much every major company in the world disagrees with you on that.

The chat agents, and the search assist function of all search engines, which I'm sure are annoying for you, is not all there is to AI. The projects going on out in west Texas where government and private partnerships will be spending 26 trillion dollars (that was not a typo) in that area alone, they also disagree. AI gets us from point A to point B way, way faster just like cars on highways vs horses and the dusty trail. If youve ever used AI, trained models on GPUs, deployed them, you'd know exactly what im talking about. If you gave me 5 minutes I'd change your mind. It makes you rethink a lot. Dont get mad im not throwing shade at you. You can have an opinion "AI is a waste of money and resources" or you think its stupid and making us stupid, but it has incredible value. We are just scratching the surface. Hell, chatGPT3 was only developed to be a language model (talk to humans and generate content/answer questions. To be good at language. It started doing calculus and writing code on its own. No shit...We're like a 4 y/o behind the wheel of a Lamborghini. Honestly we need to pump the breaks and slow down. It may not be as sexy as sending a man to the moon on the largest rocket ever made, but in terms of scale and resources, this dwarfs the Apollo missions and the Manhattan projects combined. I'm still trying to wrap my head around it. All the richest billionaires on earth are all doing the same thing, hyperscaler AI projects. Its unheard of and its weird. Everything is hush, hush but of course we all know what its all about. AGI, An intelligence on an order of trillions of times smarter than not only you or me but the collective intelligence of the earth. Whoever gets there first will write history from that point on. The reality is kinda wild. But if you think USA #1 is going to willingly take 2nd place, we're not. Just wait until we see hybrid computers that operate on photonics (light not electrons) and have quantum processing units. (When this baby hits 88mph...you're gonna see some shit) End of the day, some people find a paper clip on the ground, they throw it in the trash. Some people find a paper clip and pick locks with it, parallel some batteries together, make a fishing hook, make a tattoo gun, you name it.

[Throatwarbler Mangrove]

Pay to play?

This article reeks of advertisement. While I agree with the general thrust of the article (GPU clusters have especially demanding storage needs), what luck that the person being interviewed just happens to have a solution that no other vendor can provide!

[Roo]

Re: Pay to play?

In practice I found that GPFS has been surprisingly good at dealing with zillions of tiny files - albeit you do need to configure your setup appropriately (eg: metadata heavy for lots of stupidly small files that reek of muppet level programming). The best solution is to not to code like a muppet and pick the right tool for the job in the first place.

[Anonymous Coward]

Re: Pay to play?

Well, I think it's expected (normal) that the interviewee should promote his/her company's product (if from a company) and its unique differentiating (winning) features. Last June's non-interview B&F article on VDURA was accordingly more neutral (yet highlighting newsworthiness), imho.

Meanwhile, the io500 benchmark puts DAOS, Lustre, and Spectrum Scale ahead of PanFS (#34) at this juncture (iiuc), so maybe some more finetuning remains to be effected, or it's best for a different workload and benchmark, or it has to be tested on a system much larger than the RTX Pratt Whitney to show its true potency?

[village-id10t]

Re: Pay to play?

Also, what if their product worked and fixed the problem? Why the hell does everyone in here just assume the worst? Small files meta data = muppet programmers. Does it even occur to people that meta data has purpose? Maybe there is a reason the files are there besides "everyone is a moron". Holy hell why am I in a comments section? I'm out of here.

[GraXXoR]

Idle cycles are burning cash? Yeah but not burning cash nearly as fast as active cycles used to generate titty pics.

[Gary Stewart]

Resources down the rat hole, gains and banes but no brains

AI has lead to huge increases in the price of SDRAMs and SSDs for everybody. AI has huge power requirements that will overwhelm just about every power grid where they are currently slated for installment. All hail Colossus the Elon project and all its predecessors.

AI certainly has promise to revolutionize the areas of computing, research, and everyday use that require massive processing of huge data sets that can produce real world results. This does not in any way require intelligence which is good because AIs don't have any. But in it's current form it is also fully capable of GIGO on a legendary scale. So could somebody please find a better i.e. more efficient on all fronts and less hallucinatory way to do AI? You will almost certainly be richly rewarded.

[Anonymous Coward]

Re: Resources down the rat hole, gains and banes but no brains

In the current climate any fractional improvement that could be made to 'AI' would MAKE YOU RICH !!!

[And there is the problem in full stark reveal !!!]

There is a lot of 'running around the edges' going on BUT little change to the basic flawed functionality of 'AI'.

What is needed right now is a lot less 'marketing' and some truth about what 'AI' can really do, factoring in the 'hallucinatory issues' that have NOT been eliminated.

How long does it take for people to realise that 'AI' is not here yet ... and is always going to be 5 years away !!!

A different approach is needed, an approach that as yet is not known ... hence the 'always 5 years away' !!!

There are better things to be spending this insane and frankly obscene amount of money on !!!

'AI' is a lottery where the ticket price is 'as much as you can spend', the draw date is the 'future' and the prize is 'as much as you can imagine'.

To all the players ... Best of luck !!!

:)