Meta debuts its first 'mixture of experts' models from the Llama 4 herd Meta has debuted the first two models in its Llama 4 family, its first to use mixture of experts tech. A Saturday post from the social media giant announced the release of two models: Llama 4 Scout, which has 109 billion parameters, 17 billion of which are active across its 16 experts at any given time. Meta says it can fit …
We used to have the FLOPS argument about CPUs, then we had the pixels argument about graphics cards, now we're having the trillion parameters argument about LLMs.
So, when will we finally reach the singularity ? A hundred trillion ? A billion trillion ?
What'll it be ?
I don't care what the perceived political bias of the output the model comes up with is. I just want it to biased toward reality.
If fixing a model's political orientation makes it spout liquid shit, then
- that's not unexpected from a political point of view
- that's not unexpected from a corporate point of view
- that's not unexpected from a computer science point of view
@Benegesserict Cumbersomberbatch
"I don't care what the perceived political bias of the output the model comes up with is. I just want it to biased toward reality."
I do wonder who is running a model for a chat about politics but to each their own. As you say a bias toward reality would be far more useful.
Unfortunately, reality can be very, very unpleasant indeed. People may be racist, misogynistic, violent, bigoted, elitist, sadistic, bullying, mad, deluded and stupid beyond belief - that is a reality. Whether it's one we want thrown back raw and unfiltered in our faces is another question.
Opinions exist; they are indeed a part of reality - their content, though, frequently not so much. (And indeed, are humans' views and beliefs about their relation to the world and other humans in it not real?)
And therein lies the problem: LLMs are trained almost exclusively on the opinion part of reality.
On the other hand, we humans are, arguably, trained on the full reality... and look what we get. The problem is that (did I mention?) opinion is part of reality, and, it seems, a part we choose to weight unduly heavily. (Hence religion, superstition, quackery, conspiracy theory, post-truth and other woo.)
This post has been deleted by its author
Meta is just desperately kissing Trump's rump and stay on his good side. They've never cared about the quality of their output. For the rest of the world this means sifting through even more steaming piles of right-wing garbage and alternative truths.
Fixing it to provide a balance. Because every broad consensus needs to be countered with an unhinged rebuttal from an isolated fringe minority, else you risk being branded... dun dun dunnnn!... the Mainstream Media!
Is your LLM pumping out thoughts on socialised care programs WITHOUT providing a balanced view that just euthanising fat people is a genuine alternative? Bloody leftists!
> "It's well-known that all leading LLMs have had issues with bias — specifically, they historically have leaned left when it comes to debated political and social topics," Meta's launch post states, before attributing that to "the types of training data available on the internet."
The data on the intertubes isn't right wing enough because the sheeple haven't been fed enough alternative facts - we'll just have to fudge it for now. But give it a generation or two and our new world order will adjust their way of thinking, and the training data will gradually need less "adjustment". Of course we'll have to de-fund science and history to keep the truth safe, but I'm told that's already being done (cheers Elon).
