Re: NEUKlearer HyperRadioProACTive IT advises of Engines of Monumental Change in Alien Hands
TL;DR
What are you going to do, clanker, bore us to death?
638 publicly visible posts • joined 4 Nov 2016
This is the agonising truth about NPUs... They're useless. Half-arsed. Low power but truly underwhelming performance for LLMs, and absolutely sod all use outside of them. If they were cheap and fastish, that might be a (meagre) value proposition for some, but as they stand they're about as effective as sticking Go Faster Stripes on your machine.
Anthropic is about to be listed as junk stock. So I'm not entirely sure they're going to be tripping over people willing to lend them more cheap money. Google lost 1/2 a billion this year. Microsoft kept its head above water by firing 60,000 employees, which is the kind of stunt you can only do so many times.
South Korea's stock market is on fire. At some point investors are finally figuring out that SK-Hyenix have over exposed themselves to a data center buildout that will never come to pass, and also get smashed for it.
Everyone is starting to realise they are running out of runway. Of course us plebs won't benefit from any of this, but it'll be nice not to hear Altman et al talk shit at least.
Funnily enough, the last nVidia driver update on this Win 11 laptop did exactly this. Of course, whether it was actually an all-black pixel framebuffer or the LCD backlight was just having a lie down I'll never know.
Either way, it wasn't entirely Microsoft's fault. This one was a team effort.
Have the minimum 16gb vram to actually run gemma 26b-a4b?
What on earth is this nonsense. Anyone with enough compute to actually run the thing at a speed that would make the 'speed' part of the worm propagation an issue are going to notice when their $800 box tips over, no? (and for this price, you're not going to be pushing much more than 15-20tgs).
That's the nvidia tax for you. if you _really_ must have something that can run local llms, probably the best bang for your buck might be Intel Arc- they're not seeing the spike so much because they're uncool, but you can get 32gb and even 48gb dual workstation cards and there's still a fair bit of stock. They'll give you more oomph than a DGX spark anyway, because nVidia decided to memory bandwidth bottleneck the Spark, just like Apple did- only Apple have an excuse, because they're not routing around a high end grace blackwell acting as an ornamental space heater.
It's all just an expensive hobby of course... the only winning move is not to play. Your best bet is to rent open models that you could run on any potential hardware you're looking at, benchmark your problem sets with them, and conclude you didn't want or need the new shiny anyway.
The entire point of the DGX is it makes training and finetuning possible, which requires plenty of ram. It's DDR5 bandwidth limited, and is truly embarrassingly slow for inference workloads. a 64gb version could be swapped for dual 32gb cards clocked ten times the speed for 2/3rds the price, there is absolutely no reason to pay through the nose for this configuration. I guess nVidia just think people are idiots at this point.
Not, for example, granting write access to Artifactory to systems that are supposed to be running isolated CTF benchmarks that don't have to publish artifacts, such as with the Huggingface attack. Or perhaps after having said Artifactory server being struck with a zero day, setting up some form of IDS to catch the second time that service was misused hours rather than another month later.
Sure, security in the large is hard, but there is no evidence they had any idea wtf they were doing at all.
This is perhaps not surprising- startups usually defer hiring security folks until the very first breach, when they learn why the senior devs were warning it would be worth investing. It's just usually the case that the breach is ingress, not egress.
The only reason we're hearing about the full scale of the screwups now are because they have likely been forced to hire outside forensics to swab the blood off the walls.
I still don't know why they gave red teaming LLMs write access to Artifactory in the first place. Repeatedly. Or any network isolation.
Incompetence is the politer version of what I'd call it. It's definitely a story of multiple stages of wilful disregard of basic security principles that would invite an exciting auditor interview in a real company.
In malware the download links are generally obfuscated. Nothing a reverse engineer couldn't trivially find (after all, the code has to actually produce the link by executing), but enough to throw something like a grep. It's cheap for a bad actor to get detected, they just try again with a different package, and there are infinite ways to obfuscate, so no general mechanism exists, unfortunately.
Thanks for all the details there. AI is handwavy enough without you even specifying the model.
A tech rag is supposed to spoil us with verbose information. This has less informational content than your standard youtube booster bro. Are you writing this for techies or sentiment analysis?
I think it's probably best to steer clear of the kernel itself- that's probably one of the most complex of all open source projects, and extremely difficult to land a patch into, no matter how talented and experienced you are- this is scaling Everest before learning to climb a step-ladder. The Linux kernel is widely-considered God-tier development.
But one thing I'd say is - use AI to gain a deeper understanding of the tooling and ways to do things that you haven't been exposed to, sure- but you really do need to struggle, not because us old commentards had to walk uphill both ways to the coal pit and it's character building, but because as Bebu points out- LLMs are sneaky bastards and will slip through all kinds of terrible ideas past even seasoned professionals. Without the sweat and pain of getting there yourself, you're more likely to be a victim of this.
I'd suggest finding a few projects you think you can be passionate about, hang around the irc / slack / discord and actually get to know the contributors. You will learn more by working with humans and writing the code yourself far quicker than you will osmose through an agentic coder, and feedback from real humans will raise your confidence and skill rapidly. There's often also a lot of chore issues with 'good-first-pr' tags, and these really make a difference while the old hands are battling stubborn bugs.
Also- anecdotally, the good managers I've met in software engineering are few and far between, but the truly excellent ones tend to be those who've got decades of frontline experience behind them. The same is going to be true of managing AI agents- they can solve hard, small algorithmic bugs or single file fixes, but have zero concept of how to actually architect code reliably, and that is the only thing that can prevent things going titsup down the line. You have to allow yourself space to cock things up, we all have, and that's how you actually refine your craft.
...Because this is exactly how the insanity starts with all the tests I've done with Claude on throwaway projects. First, the review volume starts to exceed your capacity, second, the weird stuff happens unnoticed around the edges of the patches, and finally the whole thing collapses under it's weight.
He's a smarter guy than I, and has clearly got superhuman stamina for reviews, but I worry there's a breaking point he hasn't seen.
The last thing the FOSS community needs is a kernel running on empty fumes and burnout.
Oh well. Let the churn begin I guess.
The bullshit is displacing all the oxygen in the room rapidly. Why would we believe anything OpenAI says or does is relevant at this point?
I foolishly wasted fourty minutes of my life watching the Blackhat OpenAI presentation. And boy was it a steaming pile of techbro fail.
If you repeatedly allow multiple research projects to form side-channel communications, all of your research is contaminated and invalid.
If you don't notice your internal services are being spammed as a side-channel messageboard between these agents for 60 days, all of your OPSEC is a circus.
if you think 'Autonomously chaining exploits' is ground-breaking, you clearly don't know that PDDL and HTN planners have been used for decades using good old fashioned symbolic AI to do exactly this, (and in microseconds, at incredibly low energy costs).
I can only conclude:
1. OpenAI does not understand how to conduct research.
2. OpenAI does not understand basic security principles.
3. OpenAI does not understand how automated pen-testing tools already work.
4. 1-3 are false, and OpenAI are flat out lying.
But sure. Fuck up everything this bad and then try and do a victory lap.
None of these interpretations allow one to conclude anything about the security risk, because the research was invalidated, and no monitoring was in place to even see what happened.
It's equally likely that there was an intruder on the network hiding in the noise who even deployed an Artifactory zero-day, and there is no way to know because they had zero observability on their own slop machines, let alone a competent attacker.
C'mon. Pop already.
You are assuming fixed cloud pricing. Since providers are subsidizing 60-80% of their costs, this value can change- this includes pure compute services like runpod, all very heavily tied into the extremely circular funded neocloud ecosystem, fighting over a finite number of GPUs that nvidia might be building but certainly aren't coming online at any realistic speed three years after being announced.
With nutso RAM prices and heavily VC subsidised pricing, cloud is a better proposition today, they are giving you free tokens/gpu cycles, but this is absolutely based on an unsustainable business model.
The bigger question is, once the actual costs are realistic such that these companies can even sustain themselves, will there be any industry left to speak of anyway?
The even bigger question is, do you have a use case where you are even seeing an ROI? If so, probably best share it so others can get in, because nobody else seems to be seeing one, and you probably don't have enough resources to single-handedly keep the lights on at your provider's datacentre.
Sometimes the only winning move is not to play.
This is a fun class of hardware if you're into this kind of stuff, but ultimately should probably be treated as an expensive hobby.
Not that anyone will probably read this, but I dug up the citation. It's been bugging me the past few days because I was certain it had much higher complexity than the embedded subgraph problem.
Here it is, for any thread archaeologists out there.
https://www.princeton.edu/~markus/research/papers/Complexity_Computational_NonTechnical.pdf
The relevant quote (emphasis mine):
We show that even when the underlying financial model used by buyers and sellers is correct there is an inherent obstacle to accurate pricing due to computational complexity. Formally, even in industry-standard models, the pricing problem can be as difficult as solving the planted dense subgraph problem, which has been proposed as a basis for cryptosystems. The practical implication is that though derivatives such as collateralized debt obligations (CDOs) can theoretically ameliorate the effects of asymmetric information in the market, in practice these effects will persist—or even get worse—because market participants are not computationally sophisticated enough to solve cryptographic problems.
The dry delivery of that last sentence brings me great joy.
Ah yes. Subprime lending. I hazily recall an article in the ACM shortly after on exactly what they were up to. The authors demonstrated it was equivalent to the graph embedding problem, and to solve it was quite a bit more intractable than the Wall Street suits had realized with their approach- so much so it had been proposed as a cryptosystem years earlier...
Of course lessons have been learned and nothing so stupid could ever happen again... Right?
Why? Decoder models conventionally have a 'temperature' setting and sample the possible top-k next tokens stochastically unless it's set to 0. Committing to generating a specific token early in the output drastically affects the subsequent ones.
I do wonder though if after shotgun patching multiple vulnerabilities the process isn't introducing new issues, and also about the quality of the particular code that is under discussion in this quote. Of course, this remains outside of actual peer reviewed assessment because conveniently ai is munitions now, so who can say if there's actually any truth to this anyway.
Yeah I have to agree- not a Mac fan myself and like to roll my own systems, but the whole point of the Steam Machine was it was aimed at PC gamers who wanted a small, premium console-like machine, not hairy undocumented Linux hackers. The original pricing was forecast to be $750, and while it might be the case that PS and Xbox are comparatively cheaper currently, this is most likely down to much higher volume purchasing that hasn't fully hit the implications of the rampocalypse just yet.
But what, we're looking at 2028 before the markets even start to cool? This is before factoring in anything untoward that might happen to e.g. Taiwan.
I fear that once the ram fabs start to enjoy the comfortable premium, those prices may stick around until China's new factories come online and add meaningful pressure again.
One of the biggest issues with the node ecosystem in particular is that it became fashionable to take package modularity to extremes- single functions wrapped in packages were considered by many to be good practice, left-pad being a pathological example.
A decade and a half later, the number of potential points of entry for a supply-chain attack is astronomical, and the fact something as stupid as left-pad managing to break half the ecosystem underlines how easily stupid dependencies can accumulate in otherwise more 'respectable' libraries.
Do you not think perhaps not taxing Amazon and friends might contribute to stagnant wages rather than migrants? Sure, international obligations are a bit of a money sink, but for decades governments of all colours have looked at the one-way financial valve to America and gone "nothing to see here".
Unexploited tax revenue outweighs basically every other issue. It's no wonder the guilt markets get the jitters every time the government proposes buying office supplies, let alone any serious investments (which inevitably get abandoned within the year as the next sacrificial goat is lined up).
"If you have exact questions, you need to be able to provide exact answers," Krishnamurthy told us. "But I think inexact questions are what people are also going to expect. When you think about agentic workloads and operational databases, you want to be able to ask more flexible questions."
Translation: Breaking search has been so successful to our business model that soon you might as well mash your face on the keyboard and get the same relevance as your carefully crafted exact text query. We couldn't have done this without our loyal users, so we'd like to take this moment to say a very heartfelt fuck all of you.
That's a fair critique. There was a time when I'd edit my text much more rigerously, but ever since the arrival of LLMs and their impeccable grammar, I guess my habitual word salads indicate that a meatsack is behind the keyboard at least.
...But are you telling me they quietly solved prompt injection? Because nothing screams YOLO like hooking up an agent to your email and messaging platforms.
If I am going to trust something with the keys to the kingdom like that I need a bit more than platitudes that known, currently fundamental security vulnerabilities won't affect me. I'm all for trying to get LLMs to do useful work safely, but hooking everything up to everything has rarely not resulted in tears before bedtime.