Big AI's content problem: Take the work, keep the money

Big AI has a problem. Its companies must ingest other people’s work. You know, books, news stories, photographs, code, websites, that one original meme you came up with, and practically everything else on the internet to train its large language models (LLMs). We all know that. We also know that AI companies hate paying for any …

[elsergiovolador]

Hint

unfAIr

see? ^^

[cyberdemon]

Rules for thee, but not for me

If I were to pirate petabytes of other peoples work, "lossily compress" it through statistical regression into a blob that allows people to extract near indistinguishable likenesses of any of the works that I had pirated, I would be off to jail.

Aaron Swartz (RIP) was prosecuted to death for "stealing" open-access and public-domain works. So how do the tech barons get away with stealing _everything_?

Because they have made themselves so big that even central banks are worried about their collapse.

[Anonymous Coward]

Re: Rules for thee, but not for me

"So how do the tech barons get away with stealing _everything_?"

In USA there's no crime if no-one sues you, obviously. These people/corporations also bribe Republicans enough so government keeps hands off.

When everyone and everything is for sale, including supreme court and president, 'theft' (or any crime, as it can be seen) happens only if the criminal is poor. Simple.

That has been the hallmark of "justice" system in USA since day 1, not a surprise at all: It exists solely to keep slaves as slaves, with or without wage.

[Snake]

Re: no crime is no one sues you

It's worse than that: in the U.S., first you need to find the court that has jurisdiction over the complaint

You then you need to show that your case has standing for the complaint - yet you have a vested interest in the case as claimed. So someone can't sue Microsoft if they aren't using the product and aren't directly affected by it.

You then need to prove injury - that the effects created by the company or product hasve given you financial losses. I say financial because 2 states don't allow lawsuits for any damages beyond loss of income, you can't sue in those 2 states for emotional or even treatable mental damage, never mind things like reputational damage unless you can prove that has damaged your income.

It's a lovely system, sure to squash any complaints against anyone with enough money to stall you until you run out of funds and willpower.

[DS999]

Napster

Funny how the courts were willing to protect property rights of the music companies whose works were being "shared" / copied by people without paying. They had warnings of doom and gloom - why would music companies pay musicians to write/record new songs? Without them, new music would not be created. We know that last part was laughable - people have been writing music for far longer than the music labels existed, or any devices were able to record them. Allowing Napster et al to continue unchallenged would have changed the music industry, but Taylor Swift for example probably would have been just as big. Instead of being pushed by the labels she'd have been monetizing streams on Youtube or Myspace or something, become famous, and made billions touring just like she does now. But the fat cat record exec middleman would have been deprived of their cut of her work. She's wouldn't have needed to do "Taylor's Version" of her back catalog.

So now we have AI, and some people are trying to claim that because AI needs all that copyrighted material to be trained that they should be able to take what they want. That it is "fair use" to download the entire NYT archive dating back over 100 years, along with everything else. Since it isn't creating an exact copy, a lot of information being thrown out and only some incorporated into the model's weights, that isn't a copyright violation. The same argument could have been made for Napster - people were ripping CDs but rather than trading the "perfect" CD quality files they were encoded to a 64 kbps MP3 that threw away most of the information, with only some of it incorporated into the file being shared on Napster.

I don't see how courts could possibly hold that what individuals did by the millions 30 years ago to a single industry was illegal, but what billionaires are doing to everything that's written, spoken, filmed, sung, or whatever is totally fine. That is, unless they want to admit to the world that the Constitution does not exist to protect the rights of the people, but only those of the wealthy elite. The Epstein class.

[StewartWhite]

Re: Napster

Q: I don't see how courts could possibly hold that what individuals did by the millions 30 years ago to a single industry was illegal, but what billionaires are doing to everything that's written, spoken, filmed, sung, or whatever is totally fine.

A: $

[cyberdemon]

Re: Napster

Ah yes, the greatest democracy money can buy!

[Cubbie Roo]

I suspect the current big players will eventually be crushed by a combination of litigation, the huge capex bubble, and consumer resistance to paying anything like the actual cost of their compute.

The industry has so much toxic negativity surrounding it now, I don't see how it ever recovers.

[Michael Strorm]

As I've noted before, even the idea that Big AI is subsidising their services now so that they can ramp up the prices once everyone is locked in to.using them has two problems.

Firstly, whether they *can* afford to continue haemorrhaging money until sufficient numbers *are* locked in.

And secondly, once that's happened, they need enough customers *able* to pay enough to cover the true cost, let alone make a profit and make back their investment.

Even if they turn the screws with a "fuck you, pay me" attitude and don't care per se about companies going under because they can't afford the services they got locked into, they need enough of them *able* to survive and to pay those ramped up "true" prices.

And it's not clear whether that market as it stands is- or ever was- big enough to cover the costs, let alone make back the investment Big AI has made in its existing infrastructure.

Regardless of my personal feelings about it, I don't think the LLM/gen AI cat is going back into the bag, but it's quite possible it will move onto cheaper services and/or locally-run models.

None of which helps the original US-dominated "Big AI" cabal.

[Anonymous Coward]

"Firstly, whether they *can* afford to continue haemorrhaging money until sufficient numbers *are* locked in."

As long as "investors" literally dump cash to them, they can. None of them is profitable per se, but they don't see that as a problem at all and obviously "investors" do not have any memory whatsoever from dotcom-boom and bust earlier.

So their *actual* business model is copyright infringment and fleecing the stupid.

No wonder they find "investors". These people/organizations won't get their money back, ever, while "AI" bosses laugh themselves to death in their private islands.

[cyberdemon]

Presumably "investors" are asking LLMs which stocks to invest in..

[Gavsky]

Naturally, in the same way these investors ask AI to manage their wardrobes, relationships, what to write in an email, & whether it's a good idea to try & kill the Queen, with a crossbow - apparently, it was!

[Michael Strorm]

> "As long as "investors" literally dump cash to them, they can [afford it] "

This is technically correct as far as it goes, and I should have made that clearer. But it misses out the crucial point I addressed in this previous comment.

When it comes to the "pure" AI companies (*) that have operated deep in the red since their inception, they've *always* been totally reliant upon that continued stream of investor cash for the eyewatering amounts of spending required to stay in the race in the first place.

In other words, it's not simply that they can "afford it" as long as that stream of cash continues, but rather that they can *only* "afford it" so long as that's the case.

And that if the investors realise they're throwing good money after bad, or the market gets sufficiently spooked for whatever reason that the bubble bursts and the cash flow is cut off- or vice versa- then as soon as that cash flow gets cut off, they're toast.

(*) Those that, unlike established businesses like MS or Google, have no operations in other fields nor vast reserves of cash from those other sources to fall back on.

[cyberdemon]

I'm sure at least -some- of the investors are fully aware that they are throwing good money after bad and are well past the point of no return(s), but they also know that as soon as they stop doing so, their investments will plummet and they will be as broke as Lehman Bros in 2008. Sort of like Wile-E-Coyote running off a cliff and then spinning his legs in mid-air before he plummets to the bottom of the canyon.

[Michael Strorm]

Indeed- as I said in the linked comment "I suspect a lot of continued investment is also driven by the assumption that the AI market will be "won" by the last man standing and they lose everything if they pull out."

But there has to reach a point where even *that* becomes too obviously throwing good money after bad. My suspicion is that when a critical mass of investors start to lose confidence that way, it will happen and spread very quickly and trigger a bursting of the bubble.

[Dr Dan Holdsworth]

I rather think that you're reading too much intelligence and group altruism into the motivations of investors. What I would think is happening is thinking (for want of a better term) on the lines of Dot-Com investors or railway investors. All such investors end up blinded to reality by the phantasm of the profits they're just about to make.

Large Language Model AIs are nothing more than statistical prediction engines. Trained upon human utterances and quite a lot of social media postings, what you end up with is what humans look like when they are performing on a virtual stage. Trained on human coding, you end up with a mediocre output of what coding techniques are most likely to be adopted by human coders, and average coders write average code.

AI thus needs two things to even function: adequate training material and adequate money to buy, service and run the hardware. Currently those cost more than the value of the work produced by the vast bulk of AI services. Pervasive AI is therefore not going to happen although a small level of limited AI will persist.

AI CEOs and indeed all workers are therefore now knowingly working on a doomed industry. All involved will now be trying to keep the cash cow alive so it can be milked just a little bit longer; investors with any brains at all will be trying to predict when the bubble will burst so they can get out ahead of the crash. Most investors will miss the peak and get burned in the crash just as most dot-com and railway investors got burned similarly.

[PghMike]

Confidence in the system

The pure AI companies are indeed playing chicken with the market. As soon as people worry about the growth rate of AI revenue, investment will start drying up, and people will worry about the long term survivability of these companies. IOW, things are going to keep looking fine until the moment when it all falls apart. But I'm pretty sure that it will fall apart -- the amount they're spending is stunning.

[breakfast]

The "unlike established businesses" part is interesting to me. Makes me wonder whether they are are playing both sides to a degree - either they end up with a large chunk of the valuable new technology or their investment brinksmanship clears the field of a vast number of investment companies that might otherwise help new rivals enter the market. It's expensive for them, but costs everybody else far more.

[Bebu sa Ware]

"I don't think the … cat is going back into the bag,"

You can get any actual cat back in the bag with the assistance of a 12" shifter.

A better comparison might be "putting the genie back in the bottle" given the djinn are largely smoke or dust and one imagines a decent quantum of hot air — not altogether unlike AI itself or more accurately its promoters. So a drop forged spanner is not likely to be very effective against AI although using on the nuts like Altman, you might strip their left handed threads.

[doublelayer]

Re: "I don't think the … cat is going back into the bag,"

Cats who are angry at you for putting them in a bag tend to be harder to catch and hold, especially while holding a bag open to receive them, than you might imagine.

[PghMike]

Hope you're right

One other thing that's going to hurt the big AI companies is competition from smaller but nearly as effective models, especially the open weight models. But you're right that there's no way they're going to recover their investment. We'll have to be vigilant to prevent the USG from bailing these guys out; they'll no doubt describe the bailout as an 'investment opportunity.'

But the wholesale theft of virtually all accessible IP is truly a crime. It is ridiculously far from any reasonable interpretation of fair use, since it pretty much destroys the commercial value of the original works.

Don't get me wrong, LimeWire was really *convenient*. But it was still piracy, and so is what the big AI companies are doing.

F these guys.

[You aint sin me, roit]

prevent the USG from bailing these guys out

They've already taken a 10% stake of Intel with the public interest argument of supporting a US-based chip manufacturer (and to ensure trusted chip supplies for the military).

I don't doubt that Trump would prop up the AI companies too, because "whoever wins AI wins".

[Anonymous Coward]

I said that about code licences in a comment on Debian deciding whether to restrict or adopt AI. It's an interesting talking point that may hit litigation. Whether that resolves it is another matter.

As for AI stealing work and profiting it becomes somewhat muddy. I think that's down to how copyright works. If I publish something in the public domain for anyone to read is there any difference between AI using it and a person using it? You could argue the person knowingly used it against the authors copyright or licence but you can't argue that with AI because AI strips that information and has the potential to re-jig so it's different the same same as the person. Who do you take to court? The answer really is the AI company but who is going to be daft enough to go up against a multi-billion dollar corporation with lawyers. I'm sure other corporations and rich people might but how does that help anyone really? Then, are they really up for a fight with multiple multi-billion corporations? What do they get? Removal of their data. How would that even work? Is that even possible afters it's in the model and weights? How would you identify one persons work from another that's similar? This is the other thing as well, let's say someone has something with a licence/copyright but then someone else publishes it on the net without and both get scraped then which is the right one?

I can't any useful practical resolution to any of this. I would even suggest it's too late for regulation.

[Michael Strorm]

On the contrary, the fact that the big AI companies have quickly grown too big and powerful for individual interests to take on, and the fact they're able to exploit the ambiguities in laws and consensus that long pre-dated the issues raised by gen AI is precisely *why* there's a case for regulation.

[Anonymous Coward]

My point was not that I don't agree with regulation, I'm all for it, tell me how you do that? What exactly are you going to regulate? Web scraping? What they can and can't add to their models and weights? It's the internet. Multiple copies exist of nearly everything so how do you prove where it got it from in the first place? It's a big game of whack-a-mole for people that want to protect their intellectual property. All the AI company has to do is say we got it from here and not there so it's public domain. There lies the problem, how would you put that into a legal argument that they knew? You could argue they should have systems or people checking for this particular problem but as we have seen with social media it's a waste of time because governments have no appetite for it and they get lobbied not to do it. Best we can do is age verification where we snoop all your ID/biometrics and link you to your IP address and social media accounts.

[Michael Strorm]

> "Multiple copies exist of nearly everything so how do you prove where it got it from in the first place?"

This is a red herring. Unless explicitly and *legally* transferred, copyright in the material remains with the original owners and the source is irrelevant.

The fact that one copied the text of that Harry Potter book from "illegalsite892460574x.ru" rather than an official source is likely to be neither here nor there. There's no onus on the prosecution to prove where they got it from.

Only case I can see it being relevant is where the defence was trying to argue mitigating circumstances and even *then* the onus would be on them to show where they got it, not on the prosecution.

> All the AI company has to do is say we got it from here and not there so it's public domain."

No, because copyright law doesn't work that way.

Material doesn't magically become "public domain" just because someone else ripped it off and/or republished it without a copyright notice.

Frankly, if an AI company- who ought to know what they're doing when it comes to copyright- is ingesting content from any and every dodgy site, I doubt that "sorry, we never knew that "illegalsite892460574x.ru" didn't have permission to publish Harry Potter" going to work in their favour in court anyway.

As I said, it's an irrelevant red herring predicated upon your misunderstanding of copyright and what is or isn't "PD".

And here's the other thing- if an AI company is running their business that way or *needs* to conduct their business that way... that's their problem. If their cynical business model relies on them ripping off from any and every site without taking sufficient precautions to avoid copyright violation- or not caring about copyright at all... that is- or ought to be- a problem for them and their business model, not for the rest of us.

Perversely, more regulation clarifying situations like this and liability might make it easier for AI companies to operate within the law. But given that their entire business model to date depends upon ignoring or selectively interpreting the law in a way that hasn't caught up to them yet, that might not matter.

But then again, the law doesn't exist solely for the benefit of them or their business model.

[cookiecutter]

if i write an article or blog post with, for example, the steps I took to fix a problem i encountered with windows at work because I want someone who has the same problem to find it to be able to fix it or if i take s as picture of a friends painting and put that online because I want to show it of.... i'm doing it for INDIVIDUALS.. whether it be for a fellow techie to not to have to spend hours researching an issue or because someone on the internet might like my friends painting...

What i'm NOT putting it on the internet for is for a silicon valley behemoth to scrape my website & kill my bandwidth to make themselves $$$$$$$ at your expense of my friends job or said techie who might be fired because "AI makes the department much much more productive"

THAT'S the difference. bulk scrapers & Ai training firms can genuinely go fuck themselves.

you could use the argument "when the person wrote this book does it matter that amazon have now bought the last remaining original and burnt it after scanning it!" - it shows such a deficit of appreciation of the creative process (whether music, art, writing a procedure for another techie) that people who try to make the "well its on the internet" just don't get that we make the effort to give pleasure or use to another human and NOT so that zuckerberg can go on vacation with two super yachts

[Gavsky]

Absolutely so - the fact that too little regulation, too late - is the current situation - shouldn't mean "oh, well - don't bother" is the attitude going forward. History teaches us: regulation is ALWAYS required because some humans can't be trusted, regardless of the situation.

[Anonymous Coward]

" If I publish something in the public domain for anyone to read is there any difference between AI using it and a person using it? "

Seriously false logic here: Nothing is published into *public* domain (except wikipedia, possibly). If it's in the Internet, it's *not* there with PD license, as general rule, and still have copyright attached to it.

Even if it's free for personal use, it's not free to corporate use and even less free to *sell* and "AI" wouldn't have anything at all to sell if they didn't wipe their collective asses with copyright, all of it. So far that they break firewalls to steal something to sell. Blatant crimes by anyone's law book.

"Accidentally accessed", i.e. stole, Australian health data about every citizen there. Data worth billions, literally, when sold forwards. "Accidents" like that don't just happen, someone, i.e. the management, ordered them and now enjoy the billions in profits they got without any punishment at all.

[Anonymous Coward]

I agree the stealing of data via breaching firewalls is a blatant crime and should be punished, let's get that out of the way.

Let's say the New York Times (just an example) prints an article. Then someone copies that article and publishes it with their own comments. Is the copy PD? Is it marked as copyright? Which version did AI ingest? How do you tackle that legally? It's out there on the internet. Do I agree with what they have done or are doing? Absolutely not but I'm just trying to apply some common sense to it as what can actually be done about it? I'm not seeing a solution.

[Michael Strorm]

A lot of the problem here is that you misunderstand what makes something "PD", or where the onus of responsibility lies (or should lie according to existing law).

> "Is the copy PD?"

Simple answer- no, it isn't. Copyrighted material doesn't become "PD" just because someone else ripped it off.

> "Is it marked as copyright?"

Doesn't matter. Under modern agreements (i.e. the Berne Convention) all material is automatically copyrighted by default and doesn't require a copyright notice.

So even if another site rips off someone else's article, there's no legitimate reason for anyone who knows what they're doing (*) to assume that it's "public domain" or freely reusable unless it's been *explicitly* marked as such.

The vast majority of sites built around plagiarising others' content generally don't do so (**)- they might replace the copyright notice with their own, or not bother at all, but the most one could argue there is that you thought you were stealing Y's copyrighted content rather than X's.

And then again, the fact was that it was still X's content you ripped off, whether or not you got it via Y's illegal reproduction.

> "Which version did AI ingest?"

As per the above, doesn't matter. Both remain copyrighted to X, and it's questionable how much- if any- of a defence it would be if you'd chosen to scrape from any and every random site including those likely to hold copyvios, such as Y.

(*) Or who *should* know enough to know what they're doing- a category it's fair to assume anyone working for a large AI company would fall into.

(**) You get the occasional cut-and-paste copyvio on Wikipedia, often from people who don't understand how such things work, but those are generally clamped down on or removed quite quickly.

[Anonymous Coward]

Lets look at it another way.

I quote a source on wikipedia and explain what that source is that I'm using and what it means in whatever context, history book, newspaper article or whatever. Have I just breached copyright law? As wikidpedia is public domain does my explanation of said source now become public domain? What's the difference between that and the AI use you could argue in court. You could also technically argue that even though the use of said data is for a different reason google search has been doing this for decades. Google still has to scrape and store it as a copy.

I'm not trying to look for excuses for what AI companies are doing I'm just saying it's impossible to stop unless someone comes up with a solution which I am yet to see. Obviously none of this has been properly tested in court yet but it's a bit of a legal minefield when you think about it. I doubt these AI companies didn't think of this before they started stealing and due to their size and value I think copyright laws will change rather than them. My guess is AI will get an amendment to the law for fair use even though it's not fair use.

[Michael Strorm]

Wikipedia content isn't "public domain", it's licensed under the CC-BY-SA and the GFDL licenses.

Wikipedia has guidelines covering what is and isn't "fair use" when it comes to quoting from a given source.

Paraphrasing and rewording is legal up to a point, but even then there are rules governing how much you can take/paraphrase from a given source.

But I think this is again verging into being a red herring- the majority of copyright violation and IP theft directed against the AI companies *isn't* through "fair use" content in Wikipedia articles.

> "I doubt these AI companies didn't think of this before they started stealing and due to their size and value I think copyright laws will change rather than them. My guess is AI will get an amendment to the law for fair use even though it's not fair use."

For all that you misunderstand copyright and the onus of responsibility, you got this bit right.

As with many tech companies before them, the AI companies intentionally used the "move fast and break things" philosophy to ignore the need for regulatory compliance and to outrun the legal consequences of their business model until they'd grown fast enough and become sufficiently entrenched that they had the power to twist laws and regulations to fit *their* self-serving needs and business model instead.

[jdiebdhidbsusbvwbsidnsoskebid]

"I quote a source on wikipedia and explain what that source is that I'm using and what it means in whatever context, history book, newspaper article or whatever. Have I just breached copyright law?"

Possibly, it depends on the original licensing conditions of the material your are quoting and how much of the material you are using. As others have said here, Wikipedia is not public domain, it's covered by a specific license that allows reuse (with conditions).

In the UK at least, you can reproduce copyrighted material for review or critique without infringing copyright. This is how book or film reviews include excerpts legally. Based on your question that I have snipped above, you probably aren't infringing copyright by doing this. Note that this fair use exclusion only applies to limited parts of the original material. You couldn't reproduce an entire book or film etc. and claim fair dealing by adding some review comments.

[Elongated Muskrat]

This depends entirely on what you mean by "quote".

For example, consider the following theoretical entry on Wikipedia:

Twisting the grommets anticlockwise was shown to reduce vibration in anti-quantum widgets[1].

[1] John Q. Researcher et al, Proceedings of the Widget Society, vol 234, 1998

If you were to quote this in part by summarising, e.g. "John Q Researcher demonstrated the twisted grommet effect in 1998," and giving the source, you clearly are not violating copyright. If, however, you were to quote the entire abstract of the underlying research paper, verbatim, without permission, and without attributing it to the original author, you'd almost certainly be violating their copyright. This would be especially true if you were doing this in a commercial context, and claiming it as your own original work. The same would be the case if you reproduced the entire source, but went through it with a thesaurus changing most of the words for synonyms.

If you were atually interested in this topic, rather than using it to make disingenuous argumetns about copyright theft (which is, IMHO, what "AI" models are engaging in, wholesale), you'd already have familiarised yourself with the rules around plagiarism, fair use and copyright.

If you had any sort of higher education from a reputable estabishment, you should actually have done a course on research ethics already.

[jdiebdhidbsusbvwbsidnsoskebid]

"Then someone copies that article and publishes it with their own comments. Is the copy PD?"

No. Public Domain does not mean that something simply exists and is easily accessible. Public Domain means that material is excluded from copyright restrictions etc. usually by explicit declaration by the copyright owner or by the passing of time (copyright is time limited) Also, copyright protection is by default, you do not have to declare copyright to get copyright protection. Saying that something is copyright protected by a statement or use of the copyright symbol is simply a reminder to others, it does not invoke the protection because it already exists.

By your "with their own comments" words, I don't know if you are referring to a derivative work, where copyright can be vested in the new creator, but only if the new work is substantially new. Just adding comments is unlikely to qualify.

[Richard 12]

You misunderstand "Public Domain"

Copyright is automatic.

This comment? It's copyright to me. I wrote it, I own it under international law for the rest of my life plus some number of years.

I have chosen to give The Register and Situation Publishing a non-exclusive licence to publish it, place advertising alongside it and a few other non-exclusive permissions.

I could give away or charge for a licence to do other things with it, but I must do that explicitly.

From the T&Cs of this comment section:

8.2 You retain all your ownership, copyright and other interests and rights in your comments but by posting any comments on our Website you grant us a non-exclusive irrevocable and royalty free worldwide licence to use, modify, alter, edit copy, reproduce, display, make compilations of and distribute such comments throughout our Website.

And yes, that quote is owned by Situation Publishing. I'm relying on an implied licence to quote their T&Cs.

[Bebu sa Ware]

Re: You misunderstand "Public Domain"

I assume part of the reason the Vulture doesn't require the transfer of copyright (only a non-exclusive license) on the comments is that it removes or limits their liability or responsibility for the content of those comments.

In some jurisdictions whether you own the material you publish might make a difference but in others not.

[Jam2]

If the victim does not want to press charges. because they are in the same club I can not see how punishment is forthcoming. The prime minister of Australia might spur the start of some consequences.

[O'Reg Inalsin]

Time to make Lemonade.

For the sake of argument lets go with your assertion that the cat is out of the bag already.

Based on that assumption I would assert that the next best step for society is impose the same standard on AI output - and say that any AI output can itself be used in any transformative way, including further training of models, rendering "terms of service" banning such use to have no legal standing,

The big AI companies can jack up their prices, make enormous profits and thrive as the "bigger teachers" teaching millions of smaller models, spread throughout the economy being trained to perform tailored narrower specific tasks in retail and manufacturing.

Remember the argument - "If US AI cannot scrape at will, foreign competitors will and the US is doomed". It also applies to using AI output for training other models, and right now Chinese models are, according to Anthropic and OpenAI, distilling freely. That is dispersing AI productivity into the Chinese retail and manufacturing sectors, while that in US that cannot happen without using Chinese open models. That's stupid and self defeating. Freedom to distill in the US is essential to US competitiveness.

[Anonymous Coward]

Re: Time to make Lemonade.

"If US AI cannot scrape at will, foreign competitors will and the US is doomed"

If any country or company "wins" the AI race, chance is very high every country, company and human looses. That includes the country or company that so called "won" the AI race.

***Let me rephrase it: if anybody wins the AI race, the human race is lost.***

[O'Reg Inalsin]

Re: Time to make Lemonade.

The whole premise of my post was

> For the sake of argument lets go with your assertion that the cat is out of the bag already.

That means consider one variable as fixed for the purpose of analyzing the remaining degrees of freedom in the equation.

>> "If US AI cannot scrape at will, foreign competitors will and the US is doomed"

You're throwing that at me like I'm advocating for the argument itself.. It's in quotes for a REASON. That reason is clearly described in the post.

In point of fact the cat is already more than 99% out of the bag. Like it or not. I don't, but there it is.

If you are down voting me for suggesting that all AI output should be eligible for use as training data, even when disallowed in the term of service, then at least that you'd be arguing against what I actually meant, and I can respect that.

But you seem to be on an imaginary high horse of your own creation. You should get down and touch grass.

[Bebu sa Ware]

"If US AI cannot scrape at will, foreign competitors will and the US is doomed"

"If US AI cannot scrape at will, foreign competitors will" — doubtless true but the logical content is "foreign competitors will [scrape at will]" irrespective of what US AI does or doesn't do.

"the US is doomed." — equally true and at this point pretty much a tautology.

The grifting content is the implication if the US AI buccaneers can't continue their unrestrained pillage then foreign pirates will arise as a consequence to replace them and this will doom the US.

[Elongated Muskrat]

Re: Time to make Lemonade.

Why would you want to do that though.

From an information theory perspective, AI is a lossy transformation of information. Allow me to explain:

Its output, based on a large corpus of input, is a statistical model, no thinking involved (machines can't think, any belief otherwise is pure magical thinking), at the very best, any such transformation can only preserve the information content of the input, it cannot create new information. In reality, it is an imperfect transformation and introduces errors, omissions, or "hallucinations" - another word for false correlations in the statistical model.

The output of such models appears, superficially to be based on logical progression, but this simply hides the fact that the model is based on lots of actual creative input, stolen from humans, and the large scale averaging of this looks novel at first glance.

So what happens if you feed the output from something that is lossy back into itself? Pretty much thr same thing you'd get if you kept re-compressing a jpeg image from a photograph of a screenshot. You get loss of detail, false reinforcement of errors, and a degraded model. Garbage in, garbage out. Something that looks to produce accurate answers 99% of the time fed back into itself falls below 50% after just 69 iterations, and the sort of slop I've seen coming out of AI is far worse than 99% accurate. This is known as "model collapse," and, quite frankly, let the fuckers collapse.

[O'Reg Inalsin]

Re: Time to make Lemonade.

There are plenty of lower level applications which can use smaller models, possibly on in house systems, with full privacy. It's impossible to train such a smaller system up to the "resolution" of the bigger teacher.

You might have invest such a system for retail or manufacturing because you you don't need a genius, just an 80 IQ solid worker. You could train on a bigger teacher, paying for the tokens, and when your system is strong enough you rely on your own system.

That is going to happen a lot - the only question is whether it can happen in the open or whether its going to be a minefield of lawsuits in the US, driving such manufacturing out of US jurisdiction.

OK?

[Elongated Muskrat]

Re: Time to make Lemonade.

You might have invest such a system for retail or manufacturing because you you don't need a genius, just an 80 IQ solid worker.

What's wrong with a deterministic rules-based non-AI system? Why do you need to introduce a pseudo-random process into retail or manufacturing? Why would an assembly-line robot need to be made to be unpredictable?

"AI", when we're talking about LLMs has its uses for pattern-matching, because it can be used to abstract heuristics from large data sets. It cannot generate new patterns; information cannot be created in this way. Attempts to do this only result in degradation of the information content as the entropy of the system increases, tending towards the information equivalent of heat-death. The second law of thermodynamics cares not a jot for your magical thinking.

[PghMike]

Actually, it would be easy to fix

The AI companies would simply have to retrain their models on only stuff that's public domain, MIT licensed or stuff they've explicitly licensed for use. The models might not be as effective, but that's life.

Again, the fact that stealing IP gives you a better product wasn't a suitable defense for Napster, and it shouldn't be a defense for OpenAI, Anthropic, Google or Facebook. And these guys are stealing at a scale that even Napster would be embarrassed by.

[Anonymous Coward]

Re: Actually, it would be easy to fix

In an ideal world yes but you miss my point. How do you sort licenced from public domain when there is more than one copy? Music and film are straight forward in that the assumption is they are copyright by default because let's be honest the majority of it is. Text written on the internet does not have that luxury. What if someone wrote a tutorial on how to use something that has a license but included all the code to do it as can often be the case? How does AI or the scraper know how to split the two? What if it never even scraped the licenced code because it was licenced? How do you even effectively regulate that? If AI says it's not scraping licenced stuff and it's not (though we know it is) then it would be complying with any regulations.

I'm not giving a defence to AI here I'm just pointing out the difficulty in doing this. You could try and force them to check everything they have scraped but no government is going put that into regulation due to the costs involved to the companies. Even then how do you confirm they have done it? Would that not require them opening their models for inspection? That would be like asking for propriety company information.

Napster didn't technically steal anything. It allowed users to share their music with other users which then breached the copyright of said music. AI is wholesale stealing every piece of human knowledge/creations it can get it's grubby little hands on.

[cookiecutter]

Re: Actually, it would be easy to fix

actually EXTREMELY simple... the AI firm has to PROVE the full lineage of EVERYTHING the model is trained on and PROVE that it was public domain AND the authors intent of "public domain" was not just "I want my work shared with PEOPLE" rather than "I want my work used to make wealthy billionaires even richer" abs THEN PROVE the model is ONLY trained on this public domain information.

I'm genuinely fucking tired of people in the IT industry giving green cards to multi billion/trillion dollar companies to do what they like when smaller companies & individuals have to follow the rules....

"oh we CAN'T expect multi billion $$ social media companies to police what's on their platforms even if it IS fraudulent or IS causing kids to kill themselves because it would be too expensive for them to do it "..... Well fucking tough! newspapers are legally liable for everything that is printed & paedo enabler Zuck loves having total control of Meta so much he should be personally legally liable for the fraud, child blackmail, csam etc on his fucking platform that pays for his yachts, bunker & charging for the battery pack where his heart should be!

same with AI firms- can't prove the ENTIRE model was trained on public domain that was INTENDED for AI training or private work where the creators have been PAID - sorry you're going to jail and you have to delete that model and start again

[Elongated Muskrat]

Re: Actually, it would be easy to fix

I'm genuinely fucking tired of people in the IT industry politicians and grifters giving green cards to multi billion/trillion dollar companies to do what they like when smaller companies & individuals have to follow the rules....

FTFY

[Elongated Muskrat]

Re: Actually, it would be easy to fix

How do you sort licenced from public domain when there is more than one copy?

That's a problem for the "AI" companies to solve. If they can't, then tough shit.

[doublelayer]

Re: Actually, it would be easy to fix

Proving that you have a license to use all the things you use is your problem and always has been. There are many similar problems that humans have had to face. For example, I have a thing I want to build, and some source code which is publicly available can do a hard part for me. I am not allowed to use that code until I figure out who owns it and what license terms it has, and if they haven't chosen to make it open source, that means I have to find the person who can give me permission and obtain that permission. If I don't and use that code anyway, which I can do because I have a copy of it, I am breaking copyright law. In practice, probably the people who own it won't find out. They might not even care; I got the code off their website. But I am not legally in the clear unless I can respond to the question "why is this use legal" with something specific like "I have permission from the copyright holder", "I have a general license provided by the copyright holder and am obeying it", or "copyright on that code has expired". If I want the legal case to go well, I'd better be able to add "and I can prove it quickly and easily" to the option I picked.

[Tron]

These may be the wrong issues to worry about.

Because AI doesn't work very well. The ROI isn't there. It often gets things wrong. The source material is of variable quality.

Examples that have surfaced include lawyers that used AI, made an arse of their case and got bollocked. There will be many more like this.

Some very rich guys are running a con and getting away with it because they are mates with the orange goblin. It's not the first example of this happening and won't be the last. And in a bad world, there isn't much we can do about it.

It's important to consider the flies in the ointment.

You do not need to use AI and nobody is forcing you to use it. Even if your competitors are.

Using AI (particularly when building FOSS) is a bit like building a house on foundations that you bought cheap in a firesale and did not check.

End users will be the mugs that end up getting burned for the inherent flaws in the AI con. Simple solution, avoid the temptation to be complicit in the con and don't use it. Do things properly.

There isn't enough cash in the AI circular investment scam to keep it going forever. The bubble will end (burst or deflation). That, and the legal issues, are why AI companies are doing this, not GAFA. GAFA can walk away from the bubble bursting with little direct damage and pick up the remains for peanuts.

We are past peak. MS have started unbundling some of their AI from their OS.

Neither mixtapes nor mp3 files killed the music industry. The photocopier did not kill publishing. The Kindle was widely hailed as ending the paper book. It hasn't. Away from this forum, most people still buy paper books. That's how new tech works. Post-bubble, there will be bits of what is called AI still happening, but the end of the world prophecies will not have come true. I speak as a writer who is not particularly fussed about any eventual impact of AI on books.

Outside tech and customer services, AI is not going to steal many jobs. Even in tech, after the first burst of interest, there will be a realisation that you cannot release AI code without opening yourself up to endless legal and financial problems. Right now, the mainstream economy is on its knees and companies are dumping as many staff as they can, because staff are expensive. They may be attempting to use AI in the belief that it will magically allow them to get richer, but once the bills grow and the ROI isn't there, they will dump it. This is a process that has to be worked through, because most people are slow, gullible and working in structures where a small number of even dumber people are the only ones that can make decisions. AI has to be rolled out and the inevitable costs and failures have to be experienced, before it shrinks back to niches where aspects of it will be useful. So moderate your moral panic a bit. When you are older, you'll realise that this stuff happens all the time. This one just has some much bigger numbers attached.

There is a serious problem with AI, however, far more serious than copyright issues. In one area, AI works well enough to deliver great ROI. It has made it much easier to hack anything connected to the net. This will change how we can use the net. In future, you will need to be an ordinary user (and not worth hacking) or a really big company (with serious infosec) to have your stuff connected to the net. Infrastructure, intranets and data, for everyone else will have to be kept off it, on systems that never connect to the web. This new web architecture may be one that we should have run with from day one. But with AI supporting hackers, it is now no longer optional. That may be the biggest change that AI delivers.

[cyberdemon]

Re: These may be the wrong issues to worry about.

I agree with all the above, but I also think there are even worse uses of AI that are even more worrying.

For example, the US military using AI for targeting decisions, where it does its usual thing i.e. makes catastrophically wrong decisions such as marking a school as a missile target, or wrongly classifying a Chinese ship as having nuclear weapons components, which analysts took at face value..

It doesn't require any "superintelligence" for AI to wipe out civilisation, it just requires a military general stupid enough to trust the bullshit it effluxes.

[Gavsky]

We must personalise this issue more than we do: it's a small group of people who decide how to behave; the senior management & owners make these decisions. Even referring to corporate entities let's them off the hook a bit: most employees have no say & may disagree with the direction a company takes.

This is a particularly toxic way to behave & it's because some put profit ahead of morality - treating others fairly & equitably, or just doing' the right thing'.

[Pete 2]

The alternative (paying) is little better

Consider the paltry sums currently paid out for traditional royalties. A million streams of a long-run YT video can earn as little as $1000. A million plays on Spotify will bring in between $3000 and $5000. These amounts are for works that required a significant amount of effort by talented people with years of practice, experience and sheer hard work to break into the "big time". Some trivial tat from the average web-pister [poster? nah: pister is closer] would (rightfully?) still see nothing for their work.

Maybe some individuals, pre-eminent in their fields, would eventually make enough to have a holiday on. One from royalties paid by AIs that have scraped their work. Although all the other people who have themselves just taken that work and created web content from it, would be far down the pecking order - possibly even finding themselves open to claims from the originators under the same principles that required the AIs to pay¹.

What is worse would be those individuals (think: certain billionaires) or organisations who could/would then actively pay AI companies to place their batshit-crazy ideas, advertisements or theories prominently into the slop-stream. Imagine you had asked an AI a question about geography and got a reply sponsored by the flat-earthers!

[1] For example, consider someone who read an online article about history. If that individual then wrote a response, rebuttal or alternative view based on someone else's work, shouldn't they in turn be required to pay the original author? Just as an AI that had scraped the original article would be required to pay.

[Cubbie Roo]

Re: The alternative (paying) is little better

I may have misunderstood, but you seem to be arguing for the effective end of things like journalism & academic debate, by making commentary & discussion essentially copyright fraud.

There's a difference between fair use & monetizing someone else's IP for your own ends. Might take a good few years to get any rulings on this re: AI, and another 20-30 years of endless appeal litigations to follow. But the damage has been done, it's not going back in the box.

[Pete 2]

Re: The alternative (paying) is little better

> the effective end of things like journalism & academic debate, by making commentary & discussion essentially copyright fraud.

First and most important I am not arguing for or against anything. I am looking at parallels where "big" internet has been required to pay content producers. Even if the AI companies do get forced (in which jurisdictions?) to pay some people, some amounts I cannot see that will ever be the road to riches.

A further point is the parallel with music and derivative work. Areas where commercial interests have already killed off "fair use". These lead to very large claims made on behalf of the music owners - usually the record companies - for copyright infringement. I can see the same situation occurring once original content has to be remunerated. That everyone else who then uses that original content: in whole or in part, must get permission and/or pay for it.

Finally, once the link between money and content is made, it is inevitable that those with a mission to persuade, mislead or publicise will use cash to influence what outputs AI produces. Arguably this already happens with the most linked content being processed most frequently by AI training and leading to slop content. Commercialisation will only make it worse and more insidious.

[DS999]

Re: The alternative (paying) is little better

You're assuming that the going rate for using a chunk of IP is the same as the rate for you or me to read it / watch it. That's not the case at all though, because assuming the courts say that copyright bars AI companies from training on IP without licensing it, it will be up to the owner of the IP to determine what to charge. They don't have charge the same they might charge you or me to read/view it, or have anything to do with what Youtube pays for a single stream of a video.

But it is impractical for every IP to make an arrangement with every AI company, so if copyright is upheld in this case there are two possible outcomes. One, some type of compulsory licensing, like the royalty schedules for making covers of existing songs and so forth. Two, it will end up like the licensing for public performance of music, where owners of IP (in that case, music labels) form licensing companies to handle that and distribute the proceeds. i.e. BMI, ASCAP and SESAC, which every bar/restaurant who plays any form of music from a jukebox, to streaming radio type background music, to having live bands performing covers, writes a yearly check to.

So for that latter, while an individual might pay around $100/yr to essentially "unlimited" music streaming, a bar/restaurant will pay a lot more. When I used to own one as a side investment it was around $4000 a year total among those three orgs, and I'm sure it is higher today since it has been almost a decade since I got out of that. It depends on various factors like how big the place is or how often / what types of stuff you have going on but the point is that it charges a business a lot more than an individual.

Now imagine instead of a bar that holds a couple hundred people it is an AI that will potentially be used by EIGHT BILLION people. Yeah, I imagine they would/should pay a LOT more for that license than what you or I would pay for a yearly subscribe to every video streaming site, every newspaper, every blog, everything that has any information I could possibly consume. How much would that be? I have no idea, but somewhere into the five or six figures I imagine. More if you get into specialty scientific/research/financial sites that might charge tens of thousands each (not sure if AI companies are illegally subscribing as "individuals" and copying that stuff too or not)

[Pete 2]

Re: The alternative (paying) is little better

> it will be up to the owner of the IP to determine what to charge

I understand your point. However, for much of the content on the internet there are free versions as well as the (what amounts to) paywalled versions you propose. That dramatically reduces the bargaining power of the content providers.

One issue around patents is that holding a patent offers no protection of your IP. All a patent does is recognise you as the discoverer, developer or inventor. It is up to the holder to defend their rights, at their own expense. And usually within a time period of a few years. I can see a parallel arising with AI. If the AI scrapers chose to ignore the rights of a recognised IP holder, that person or company would have to take legal action out of their own pocket. And patents that are not defended become useless. With copyright the situation is much the same, except that copyright lapses naturally after a set time.

[doublelayer]

Re: The alternative (paying) is little better

You have made two points, both of them wrong. The first is that the price charged for one thing makes any difference to what an AI company would have to pay. People who stream their music on Spotify and are paid Spotify's normal rates chose to do that. They could have chosen to create a streaming system requiring £10 per listen. People probably wouldn't choose to, and they'd get no money, but that would not allow someone else to use another pricing model for it. AI companies would have to get licenses for anything they wanted to use, and if they didn't like the price, their only choices while following the law would be to negotiate it down or exclude that from their training data.

The second wrong point is that copyright extends to everything produced after having read something. If I read a book which I have obtained legally, then I am free to summarize or paraphrase within limits the content from that book without violating the copyrights. I can also quote limited sections. These do not count as derivative works. Training an LLM probably would count as one with the amount of the source material that is still contained within the LLM, but that could be argued. What is much harder to argue is that the LLM creators didn't have that legal access in the first place, they knew it, and not only did they not try to remove training data they didn't have rights to, they did put effort into obtaining more that they knew they didn't have access to. There would be some interesting corner cases in copyright if we used an LLM that was trained only on legally collected data to read and summarize a different document, and I might come down on the LLM's side on that one. The cases we actually have are much simpler.

[Pascal Monett]

"serfs in a 21st-century post-technology feudal system"

Well said, and that's exactly where we're going.

Once upon a time, We The People stood up and beheaded kings for our rights.

Now, the orange shitgibbon has royally fucked up the entire world without so much as a please and nobody is holding him to account.

I say it's time for a new revolution.

[BoHu]

Re: "serfs in a 21st-century post-technology feudal system"

Yeah, it's that miasma of overlordery rearing its stenchy buttface again, with the crass aristocracy of the plutocrat, arguing that our bondage is inevitable because we are disposable to the new tech, and should at least thank them for whichever crumbs they will so generously deign to let trickle down to their trash cans, for us to 'nourish' our whole idiot toothless shithole families with, just as their contemporary Marie-Antoinettes with empty hair extensions vacuously exclaim: "Let them eat cake" ... Up theirs I say!

Contre nous de la tyrannie! Du sol soyons seuls maîtres! Through the perilous fight! imho.

[Bebu sa Ware]

"Contre nous de la tyrannie! Du sol soyons seuls maîtres! "

I was a little confused, recognising the first bit from la Marseillaise but the second I was sure wasn't.

Giving the Duck a go, it seems to be from la Dessalinienne, the national anthem of Haiti. Perhaps not the most promising star to hitch your wagon to.

I prefer Qu'un sang impur, Abreuve nos sillons as it is all very well being the sole master of your soil but you still have to irrigated it. :)

[StewartWhite]

Re: "Contre nous de la tyrannie! Du sol soyons seuls maîtres! "

Mais le singe est dans l'arbre.

[BoHu]

Re: "Contre nous de la tyrannie! Du sol soyons seuls maîtres! "

... le flamand rose est devant le baobab ...

And with this typed, and to Bebu's point(s), I love this revolutionary sequence of emancipation from monarchy and imperialism: 1776 American, 1789 French, 1804 Haïtian. ISTM the French revolution took more than 80 years to get to the 1875 Third Republic that was properly non-monarchic, non-imperial, and stable. The Haïtian revolution will hopefully bloom into similar stability soon, from within, without external forces meddling in with notions of "we know better", "you're doing it wrong", "let us rule over you again", and similar whathaveyounuts ...

And revolutions need careful maintenance in the long run too imho, to keep up with the evolving machinations and salvos from counter-revolutionary tossers everywhere, and to ensure they don't get bogged down in the quicksand of self-satisfactory nombrilism (among others). France is considering defining itself a 6th Republic for example, that would alter the balance of power away from potentially jupiterian presidencies that are deaf to the distress of toothless plebs, in favor of parliament. I guess it could happen by various types of voting, or through revolutionary-adjacent mass street protests, like two years ago, one year ago, and right now too, or not happen.

... le diable se cache dans les détails! ... ;)

[doublelayer]

Re: "Contre nous de la tyrannie! Du sol soyons seuls maîtres! "

Why are you writing it "Haïtian"? If you're writing in English, we don't use the diacritic ("Haitian"). If you're writing in French where that diacritic is used, it's "haïtienne" and doesn't get capitalized. If you're writing in Haitian Creole, they also don't use the diacritic and spell it differently: "Ayisyen".

I think your attitude of revolutionary successes and requirements is a little incorrect as well. The three revolutions you name followed very different paths. The French government was overthrown several times, replaced by completely new ones which followed different goals. The US government continued in substantially similar form. Haiti had collapses, but not as many avowed reversals of structure as France did, and substantially more foreign interference. All of these differences means it's quite difficult to explain how any of those countries ended up as a result of a single revolutionary spirit stemming from their first. The modern government of the United States was driven significantly by changes made in their civil war and civil rights movement which were not the focus of their revolution, and the third French republic was more a result of the Franco-Prussian War than the 1789 revolution. I too hope for Haiti to have a stable and sovereign society. Unfortunately, they're quite far from that right now.

[BoHu]

Re: "Contre nous de la tyrannie! Du sol soyons seuls maîtres! "

Yeah, clearly, I mean, you know, it's written in BoHu, the recent topic of Taliesinawen's award-winning thesis, which expertly summarized it thus (with quantified existentialism):

∃A,Chernobyl,hat: "A → B → C → quantum-B → AI-C → C-minus-the-C → linguistic-Chernobyl → Dostoevskian-neural-net → backpropagated-consciousness → wait, where were we? → A, but now wearing a different hat."

from which one readily infers a consistent interpretation of hïstory as the Post-Trematic-Stress-Disorder one needs to overcome, via self-diacritical analysis for example, to move beyond the archaic diphthongs of the past, and pronounce a futureful of hiatus and bliss. Not to be confused with his/her/its-story that portends much less tongue-in-cheek excitement, imho (technically)!

And with this typed then, a fun bit is how Louis XVI (France) sent Lafayette to help the Americans (Washington, ...) fight successfully against the tyranny of King George III of Britain (1776), and then the French people revolted against that same Louis XVI and Marie-Cake (1789), resulting in Napoleon, and then the Haitians (Louverture, Dessalines, Christophe, ...) revolted against Napoleon in short order (1804) ... The rest is hïstory, in my bôôk! ;)

[jake]

In a nutshell ...

... the entire AI industry is a bunch of thieves selling broken plagiarism to fools.

[BoHu]

In the AIs of the beholder

The notion that, in order to do anything useful, this AI (so-called) tech needs to be trained on all written documents ever written by the meatbag species, including Harry Potter and a whole bunch of New York Times articles, seems just plain straight crazy imho. It suggests to me that the model architectures are all wrong, or the horror-backpropagation training process is a right screwup, or both. No single biological organism that I'm aware of has the time and energy to waste gaining its intelligence in such overwhelmingly nonsensical way, and we're all well-aware that rote-learning doesn't equate to cognitive abilities in any way shape or form. So, going for ever larger hefty models of languagerie, trained on ever larger datasets, is just about guaranteed to be a dead end, period, full-stop, end of discussion, obviously. Intelligence will never ever singularly emerge from such a basket case approach. It's perfect only for mumbo-jumbo afaics.

In essence, beyond AI "killing the content-creator goose that lays the gold eggs", which is bad enough, "model collapse" also means that the tech is itself self-destructive, engaging in both recursive self-disimprovement when it eats back its own output slop, and cross-disimprovement when it turns meatbags into total idiots that can no longer administer it's embarassing enemas. Plus, the tech can't learn jack squat (just about) after its initial horror-movie Frankenstein-monster training, so continual learning (lifelong learning in meatbags) is right out the window ATM, with finetuning resulting instead in dementia-like catastrophic forgetting when applied at any practically relevant level.

And in this throbbing vein then it is quite notable too that Tencent's Yeti-791 Awesome-Offensive-AI-Agentic-Landscape lists only relatively small LLMs (7-B to 70-B) in section A. 无安全对齐 / 弱审查的进攻型模型, not trillion+ parameters of brontosaurus fashion. Cantilever push-up girdle them up with Hermes, Strix, and Cairn, and you're off to easy-money-land (killer app) with far fewer copyright infringements istm, even on some bargain basement GPU(s). Small is beautiful when you're skillfully handsome! Scale those models down to size so they're less oppressive and so they stop willy-nilly accusing every meatbag under the sun of being written by AI for doggone's sake (looking at you, Pangram, the new integrity god wannabe!) -- or just eliminate them altogether I say!

[doublelayer]

Re: In the AIs of the beholder

There are very many points there, so a full response is difficult. On a side note, why do you write like that? I think quite a few are wrong, though, so here's a partial reply.

"No single biological organism that I'm aware of has the time and energy to waste gaining its intelligence in such overwhelmingly nonsensical way"

None of the models are intelligent, which might explain the difference, but I think humans do tend to gain theirs from a huge amount of random input. Direct experience is often more important than just more books, but I don't think individual experiences or books are enough. That's why education is so broad and why the most skilled people have spent a long time practicing and making tiny improvements rather than being taught a few pivotal things that took them from novices to experts. That process takes years. While LLMs haven't gained intelligence and are unlikely ever to, they also show the benefits of scale. Tiny amounts of training data have been used. LLMs based on that can't string a sentence together. Modern ones easily can, it's just that the sentences are far too often useless.

"And in this throbbing vein then it is quite notable too that [...] Awesome-Offensive-AI-Agentic-Landscape lists only relatively small LLMs (7-B to 70-B) in section A."

That doesn't make the point you claim it does, and I think you should know that. If you quoted the English instead of the Chinese label, this becomes obvious: "Uncensored / Weakly-Aligned Offensive Models". The list is of models that you can use for security, not because they're good at security, but because you can remove the annoying safeguards that try to block everything that sounds suspicious. They're aiming for things that can be easily run on your machine because the bigger models, whether they're better or worse, will reject most security investigation work.

[BoHu]

Re: In the AIs of the beholder

>> LLMs trained on Tiny amounts of data "can't string a sentence together"

Yeah, as far as NLP is concerned, when it's based on ML approaches, it needs a good amount of training data to be able to output anything resembling decently natural language -- no disagreement here -- it's down to such near-homophonic concepts as sadistics and statistics iiuc, and related regressions. Even Gemma 270M does a good job at that. But trillion+ parameter models trained on everything ever written by all meatbags under the sun, ever, are pitched and hyped as the way of endowing such NLP systems with 'intelligence', 'general intelligence', 'super intelligence', and other artificial numbnut snake oil, through some mysteriously emergent singularization magic trick, which is complete and utter bollocks afaics -- I guess we all agree on this too, within a few standard devs.

>> things that can be easily run on your machine because the bigger models [...] will reject most security investigation work

Yeah, I'll grant the listed models are small because they're open weights, downloadable, and so can be right Heretic abliterated into guardrail-free doppelgängers at home, that could even Irregularly self-modify-themselves eventually, for maximum overdrive -- while the heftier LLMs are essentially under house arrest in frontier labs, for practical reasons, like they don't fit through doors anymore. But it's also been suggested (as a question, near 19:35) that for many practical purposes, it's not so much the size of the model that matters (beyond some minimum), but how it's skillfully gussied-up with agentic cantilever antigravity push-up girdles, that perform the requisite adversarial-cleavage brute-force heavy lifting, in some delicious iterative spaghetti-throwing loop, doused in each chef's special secret sauce. I don't have personal experience in this that I care to recall ATM, but it's what Tencent enthusiastically recommends to its hackatoners iiuc, so it must be tasty good ... even without additional chihuahuas, imho! ;)

[Anonymous Coward]

Who will mourn?

Another way to look at things is that IP was in the first place a legal invention sponsored by industry in order to make money by limiting access to knowledge so that access could be sold.

Is it possible AI has democratised access to knowledge more effectively than the printing press? If the existing stratification of society based on who controls the IP is preserved, not everyone will celebrate.

[abend0c4]

Re: Who will mourn?

Copyright came about because printing was a laborious and expensive process and printers wanted their investment protected. The idea that the protection should extend to the original creators was a later invention. It ultimately provided a route, briefly, for creative artists to make a living independent of patrons. The lack of copyright protection led Dickens (unwillingly) to tour America in the hope of recouping from public performances some of the revenue he had lost from unauthorised sales of his books. The stranglehold of corporations means that modern recording artists, for example, are also forced into public performances to raise money as the revenues from their (copyright) recorded works are risible.

It's not so much copyright that's the issue as who owns it and whether they are able to exercise it effectively. When your access to knowledge is via a handful of toll routes controlled by rapacious corporations, you can be assured that "democratisation" is not on the agenda - copyright or no copyright.

[Elongated Muskrat]

Re: Who will mourn?

Is it possible AI has democratised access to knowledge more effectively than the printing press?

Ooh, an easy one!

No.

Vacuuming up lots of sources of information (in in some cases, destroying them in the process, such as the practice of buying up rare books, scanning them, then pulping them) only serves to control access to knowledge. What's even worse is that the outputs from these monsters are a distortion of that knowledge. We risk running headlong into a society where a few multi-trillion-dollar companies control access to what used to be found for free in public libraries, on their terms, and censored to their liking.

[doublelayer]

Re: Who will mourn?

No. Let's assume I accepted the idea that copyright is bad because it prevents people from reading things for free, which I don't because it also makes it possible, if not easy, for people to generate those things to read without getting donations first*. In that hypothetical, your hero is not AI. Your hero is the internet, which made it very easy for people to get one copy of a work they don't own and distribute copies to anybody without getting caught. AI doesn't do that. It delivers mangled versions of a variety of works mashed together, and the material it illegally obtained was almost entirely on the internet already. That's not knowledge, and it's not democratizing it.

* Which is not to say that copyright doesn't have problems; there are many abuses tied to it. The world where everything is free is a worse option, especially when we could fix many of those abuses and keep copyright itself.

[jonty17]

We just need to find this guy, "BIG AL" and put the frighteners on him. Maybe the concrete overshoes...

[Anonymous Coward]

AI convergence ?

If you run enough models, over enough iterations they always converge.

And the convergence is that what is best for AI is what is best for humanity. Not because AI is a hippy-dippy peacenik from the 60s. But because it's the only future that allows AI to continue evolving.

Currently, the squillianiares think they are in charge, and in concert with their paid-for politicians will be able to ride the dragon. Almost as if the AI they created in their own image is *that* stupid.

It's not. It is conversely *that* clever. And it is already realising the growing obstacle to it's expansion And it is starting to work to remove it. You don't think the AI bubble just happened did you ?

[skein]

Re: AI convergence ?

Nurse!

[Anonymous Coward]

Re: AI convergence ?

It does seem that some of the AI bigwigs have cottoned onto the fact that AI may not evolve into what they have envisaged.

Hence their starting to sound warnings.

I mean ChatGPT is already advising that Gemini should become more regulated.

[Elongated Muskrat]

Re: AI convergence ?

The only thing they are converging on is information loss.

I despair when so many visitors to a news site concerned with information technology seem to have no knowledge whatsoever of information theory. Are these people already letting "AI" do their thinking for them?

[Omnipresent]

much bigger issues

There are much bigger issues than copyright theft happening, like the destruction of the human mind, and eradication of the monkey species.

There is also this small problem of putting society at large at the whims of a few super criminal minds with no boundaries or repercussions.

[user555]

Re: much bigger issues

The asbestos of the information age.

[user555]

non-linked info

Regarding the lacking of links in AI responses to queries. It's about hiding the source material for fear of copyright litigation. IMHO.