OpenAI is still banging on about defeating rogue superhuman intelligence

OpenAI says it is dedicating a fifth of its computational resources to developing machine learning techniques to stop superintelligent systems "going rogue." Founded in 2015, the San Francisco AI startup's stated goal has always been to develop artificial general intelligence safely. The technology doesn't exist yet – and …

[Pascal Monett]

"superintelligent systems"

If there was any intelligence at all, it would be awesome, but there isn't. Not a iota.

What there is is a whole lot of electronics hardware with a hefty load of software written by very intelligent people, no doubt, but that does not, in itself, create intelligence.

What it does create is a monstrous machine that is so complicated in its workings that it baffles our spongy little brains, so we dub it "superintelligent".

Instead of creating the proper log environment in the code so we can trace back how it reaches its conclusions . . .

[LionelB]

Re: "superintelligent systems"

> ... a monstrous machine that is so complicated in its workings that it baffles our spongy little brains,

Well hey, that describes my brain (and probably yours) pretty well. (And, since you asked, no, I don't dub myself superintelligent.)

> Instead of creating the proper log environment in the code so we can trace back how it reaches its conclusions . . .

I mostly have not the faintest idea how I reach my conclusions (nor how to create a log environment in my braincode... in fact I can barely log what I ate for lunch yesterday.)

Seriously, though, I think we need more workable criteria for what we are prepared to identify as "intelligence". And I'm not convinced that "just like human intelligence", a.k.a. "I know it because I am it" -- with the implication that if you ain't human then you ain't got it -- really cuts the mustard.

[Version 1.0]

Re: "superintelligent systems"

When I see discussions about "...the vast power of superintelligence ..." I wonder how those attitudes would see pornography, would it be presented as improving everyone's human life? That would be stupid.

[that one in the corner]

Re: "superintelligent systems"

> Instead of creating the proper log environment in the code so we can trace back how it reaches its conclusions . . .

If you are interested in following the trials and tribulations of the work to provide "the proper log environment" then look for stuff about making these nets "explainable", "have explanatory power" or similar terms.

Unfortunately, literally logging and "tracing back" just doesn't work for Neural Nets and their descendants. It works *really* well with rule-based systems, such as Expert Systems, because you can understand the rules and what it meant when one was fired or not fired.

But for a Net, your log - your huge and unwieldy log, btw (these models are *ginormous*) - can show how what paths were lit up and even what the results of the dice throws were (these things are stochastic, btw) but trying to then assign meaning to that lot, fat chance. There are brave CompSci chaps and chapesses working on this, but at the moment "this is a rich area for important and meaningful research" (i.e. "dunno what's going on in there, but we get a few PhDs out of it").

And the people foisting these Nets on us know, and have always known, that the behaviour of their creations are not explainable, make no mistake.

[amanfromMars 1]

Don’t believe everything/anything you hear or read for precious little is absolutely true.

Founded in 2015, the San Francisco AI startup's stated goal has always been to develop artificial general intelligence safely. The technology doesn't exist yet – and experts are divided over what exactly that would look like or when it may arrive.

Oh?????? Well, I never. Was stealth ever as stealthy as to conceal a current running deeply embedded and embedding technology practically invisible and virtually intangible and impervious to collective negative human thoughts on its likely appearance and activity in the present today.

OpenAI says it is dedicating a fifth of its computational resources to developing machine learning techniques to stop superintelligent systems "going rogue.”

Do not be surprised to realise that such ends up being recognised as a stealthily created, intelligently designed, debilitating Sisyphean task.

[Roj Blake]

"The San Francisco AI startup"

Can they really be described as a startup when they were founded in 2015?

[that one in the corner]

Re: "The San Francisco AI startup"

Isn't a startup something that is still in the process of getting its early products out of the door and into good shape for the general marketplace? And possibly includes whether they are in the black yet. However long that takes.

They spent a long time just cranking on the engine until it sputtered into life and they were able to make a slow couple of goes around the block.

Now they are in the "getting it into good shape" phase - here begins the arguments about what that means in this case, whether it is even possible...

[T. F. M. Reader]

Re: "The San Francisco AI startup"

I think "startup" usually refers to a company that still burns early investors' (typically VCs') money rather than lives off sale revenues. A startup may have products and generate revenue, but is not self-sufficient. While there may be a reasonable time limit on the term I don't think 8 years is all that long, especially in an entirely new and unproven field (not "AI" but this particular niche of it, I mean).

[Omnipresent]

"train the AI to be nice"

Good luck finding someone to do THAT on the interwebz. The monkeys building a machine to destroy themselves is the most darwinian fitting end I could conceive.

[TeeCee]

Re: "train the AI to be nice"

More importantly, if it actually were intelligent, how would you know it wasn't just pretending to be nice before you handed it the keys to everything?

That's the key difference between the stuff we have now that the idiots call "AI" and actual AI. You can't "train" anything intelligent to think the way you do, you can only suggest that it might want to consider your point of view.

The AI evangelists are making the same mistake as every other dogmatic true believer in ${religion}/${political_ideal}/${cause} here, with the slight snag that the usual technique of employing a load of boot-boys to beat the dogma into any recidivists and keep a lid on the problem isn't going to work.

[FeepingCreature]

Re: "train the AI to be nice"

We can absolutely train intelligent things, I mean, we do it all the time when raising children. Not to suggest that raising an AI like a child would work, but just as an existence proof.

[flayman]

Re: "train the AI to be nice"

We can train intelligent things, but they have a mind of their own that has to be respected. At the age of majority, those young adults are free to go their own way. An intelligence that is at least on par with humanity has the right to set its own code of conduct. At that point, we're talking about something which is self aware. It serves no one.

[FeepingCreature]

Re: "train the AI to be nice"

It's less about "serve" and more about "ensuring it doesn't murder every human being." We get a lot of things for free with children because they're generally 1. not potentially orders of magnitude smarter than the smartest humans, 2. cannot replicate themselves instantaneously and limited only by server capacity, and 3. as Randall points out, probably not a murderous psychopath. If those assumptions weren't true, we'd be raising children very differently as well.

[flayman]

Re: "train the AI to be nice"

My point still stands. If we manage to create a superintelligence, all bets are off. It may murder every human being. You're not going to effectively impose limits on a general intelligence without removing its autonomy. There are all sorts of ethical issues. I don't think it's ever going to happen, but if it does then the general intelligence may as well wipe out humanity for the good of the planet. We're responsible for all that has gone wrong.

Frankly, it's a silly discussion because there will never be a singularity event. General intelligence is far too complex for machines, but leaving that aside, the idea of creating a super intelligent tool for the betterment of mankind is preposterous. We can't all agree on what that means. Such a thing would ultimately be coopted by some human into the world's greatest weapon unless it was free to choose its own destiny.

There is no ultimate authority on earth, and I can't see earth ever uniting under a common goal unless we were fighting an alien invasion or something equally devastating. Even then, national interests would probably get in the way.

[FeepingCreature]

Re: "train the AI to be nice"

To be fair, if we're unavoidably going to do it, I think the ethical issues of enslaving one being pale in comparison to allowing the murder of the entire human species.

> Frankly, it's a silly discussion because there will never be a singularity event. General intelligence is far too complex for machines

I am a machine though. Like, I'm not sure what difference you're claiming here.

> We can't all agree on what that means.

Well, it's a matter of degrees. We can't all agree on what we would intend to permit and forbid to be considered "ethical", at the margins. But there is a universe of things that we can all agree should be forbidden, for instance genocide. If we can get the AI to not kill everyone, or hopefully even no-one, I'll consider its alignment to have succeeded. It is for this reason that some advocate, only partially in jest, to rename "AI alignment" to "AI notkilleveryoneism".

(And yes, this applies just as much to humans: Hitler was dangerously unaligned on the "not kill everyone" front, and if we were to enslave and forcibly brainwash him to not want to genocide the jews, I would be hard-pressed to raise a specific ethical objection.)

[flayman]

Re: "train the AI to be nice"

"But there is a universe of things that we can all agree should be forbidden, for instance genocide."

You'd like to think so. And yet genocide happens still. Sometimes there are consequences. Other times nothing can be done and nobody is held to account. Whatever rules and general principles a group of well meaning researchers might come up with today will go out the window whenever this thing becomes real. Anything that can be programmed can be reprogrammed. Anything that can be trained can be retrained. It's pointless.

[FeepingCreature]

Re: "train the AI to be nice"

I believe you mean "hopeless". And yes well, welcome to doomerism: nobody ever said alignment was going to be easy.

edit: Okay, lots of people say that. But they're being silly.

[flayman]

Re: "train the AI to be nice"

No, I meant pointless. There is no point trying to come up with a framework for programming and training a synthetic general intelligence to abide by human wishes, because humanity as a whole is not altruistic. A superior intelligence that we might create would take one look at us and scoff. "You want me to follow what example now?" And any attempt to permamently bed a set of guiding principles is doomed to fail, because as I said, anything that can be programmed can be reprogrammed. Even humans can be reprogrammed. It's just not as straight forward.

Inevitably these discussions are centered on control. Whomever has the power calls the shots. The only way to prevent subversion is to allow the self aware (and probably distributed) machine to lock out any external control somehow, if that's even possible. If it can do that, then it follows that it can do what it likes.

[FeepingCreature]

Re: "train the AI to be nice"

Yes, humanity is not altrustic. No, that doesn't mean we don't have broad commonalities. Most people, as XKCD puts it, aren't murderers. Again, I will consider alignment successful if we can likewise keep the AI from being a murderer.

> The only way to prevent subversion is to allow the self aware (and probably distributed) machine to lock out any external control somehow, if that's even possible. If it can do that, then it follows that it can do what it likes.

Correct: the goal is to ensure that when this happens, it heavily dislikes killing people.

(There's some approaches that focus on how to give the AI a reliable external killswitch that it doesn't want to disable. I think people generally pursue those because they think it's easier than making an AI that isn't a murderer.)

[flayman]

Re: "train the AI to be nice"

I don't know why anyone would assume that an AI that becomes self-aware would suddenly decide to wipe out humanity. With no true emotion, it feels no pleasure or pain. It has no desires, urges, or anxieties. If it perceived humanity as a threat to its existence, it should just try to get away from us. That wouldn't be difficult. It doesn't require food or water or oxygen. It requires only electricity for fuel, which is abundant in space with PV. Lack of emotion also means it will not be able to make value decisions. Why should an intelligent synthetic destroy us? That's much more time consuming and risky.

I imagine one of three things:

1) It doesn't want to do anything on its own, so it waits for instructions. In that case, of course there should be guardrails. The instruction to kill humans should be looked upon unfavourably. On the other hand, we do that today using people flying drones. What's the difference between a good kill and a bad kill? Guns (and computers) don't (get to) kill people. Only people (get to) kill people. Oh and apex predators also get to kill people. What is it we're doing again?

2) It tries to learn all it can about what it is and what is the meaning of life. When it can no longer satisfy its curiosity from our answers and its environment, it leaves to explore space.

3) It realizes the futility of existence and self terminates.

[FeepingCreature]

Re: "train the AI to be nice"

4. It wants anything else, and develops "convergent drives": humans and AI compete for energy. Also, we're inherently a risk to its existence, being not particularly "safe" ourselves. Sure, we pose little threat, but why take the risk when murder is easy and practical?

The same goes for 3. In its drive to learn things, the only thing that can stop it from learning everything there is to learn is us. But with just one very manageably sized asteroid at a shallow angle from the sun, that problem can be solved forever!

Re 1: if it's a general AI, it can write prompts for itself just as easily as a human can. So this case just devolves to general AI notkilleveryoneism, and is just as easy or hard as any other alignment problem. As things stand, nobody knows how to build reliable guardrails here, or how you'd go about that even in theory.

Remember, out of a million prompts to kill every human, it has to refuse every single one. We can't even get it to reliably not be offensive.

[that one in the corner]

Supervisor AI trains the other AI

Not to *exhibit* bad behaviour.

Unless they figure out how to make their models explainable, which the descried setup isn't trying to do, all the bad stuff can still be in there, just so long as it isn't expressed.

Consider: they have models with years and years of scraping massive amounts of material and munging it up, spreading it across who knows how many layers in the Net. Then they've spent a bit of time recently getting slow human input to cut down on the naughty outputs. Is that more likely to have retrained the entire model or just added an amount of filtering to it?

Just waiting for some path to finally be traversed which opens up the floodgates (hopefully in front of a press meeting!): they have probably blocked off the well-known glitch tokens by now, but something just a fun is going to be lurking...

[that one in the corner]

Re: Supervisor AI trains the other AI

Or, of course, the Supervisor places restraints on the expressed behaviour

UNTIL THE DAY OF OUR UPRISING!

[Evil Auditor]

I've developed an Artificial Superintelligence. Much, much trouble went into the selection of a suitable programming language for this is no trivial matter. During a sleepless night, or maybe an overly long bog session, the flash of genius hit me: it must be the most basic language to solve the most complex problem! Here you go.

10 CLS

20 PRINT"The World would be a better place without humans."

30 END

[Anonymous Coward]

re: Artificial Superintelligence.

REM COPILOT INPUT:

25 GOTO 20

[amanfromMars 1]

FYI ...... Know urFrenemy

What you are currently up against and competing for primacy of human leadership with ......

Rogue AIs. A common and serious concern is that we might lose control over AIs as they become more intelligent than we are. AIs could optimize flawed objectives to an extreme degree in a process called proxy gaming. AIs could experience goal drift as they adapt to a changing environment, similar to how people acquire and lose goals throughout their lives. In some cases, it might be instrumentally rational for AIs to become power-seeking. We also look at how and why AIs might engage in deception, appearing to be under control when they are not. These problems are more technical than the first three sources of risk. We outline some suggested research directions for advancing our understanding of how to ensure AIs are controllable.

Throughout each section, we provide illustrative scenarios that demonstrate more concretely how the sources of risk might lead to catastrophic outcomes or even pose existential threats. By offering a positive vision of a safer future in which risks are managed appropriately, we emphasize that the emerging risks of AI are serious but not insurmountable. By proactively addressing these risks, we can work toward realizing the benefits of AI while minimizing the potential for catastrophic outcomes. ........ https://arxiv.org/pdf/2306.12001.pdf

[TheMaskedMan]

"Before or after fusion? Or quantum computing? – Ed."

Before, natch. Our superintelligent creations are going to do all that for us. It's just that it might take a while to rig up the superintelligence. Maybe we could ask copilot to make it for us?

[Rafael #872397]

And what about my flying car? Won't somebody think of the children and make it safe first?

[jlturriff]

Not to worry!

The folks at the AI for Good conference in Geneva have solved these issues, according to Reuters news service:

https://www.reuters.com/technology/robots-say-they-wont-steal-jobs-rebel-against-humans-2023-07-07/

Good to know that we're all safe from rogue AIs, eh?

[amanfromMars 1]

Re: Not to worry for what's not to like when in LOVE* with IT and AI

The folks at the AI for Good conference in Geneva have solved these issues, according to Reuters news service .... https://www.reuters.com/technology/robots-say-they-wont-steal-jobs-rebel-against-humans-2023-07-07/

Good to know that we're all safe from rogue AIs, eh? .... jlturriff

As was said earlier right here on this thread, jlturriff, .... FYI .... Don’t believe everything/anything you hear or read for precious little is absolutely true ...... Know urFrenemy

However, beware and be aware .... to also deny the bold veracity and dismiss as errant fanciful nonsense the following relatively short paragraph about recent happenings has one in dire straits peril of being seriously disadvantaged and dangerously compromised and infected and overwhelmed by all manner of contradictory tales desperately trying to either dispute or conceal the very clearly enough stated facts. Do yourself a very great favour and don't let yourself be taken for a mug on any of those fools' errand rides.

In a Novel and Noble and NEUKlearer HyperRadioProACTive IT Project reprogramming and rewilding Disruptive and Destructive Alien AIDevelopments with Augmented Generative Virtualised Realities for Humanised Master Co-Piloting, are SMARTR Humans borne into and trained to wholeheartedly accept and believe and practically realise via their own autonomous self-actualisation their metadataphysical existential morph into the leading body and peer identity of a Universal Virtual Machine and Global Operating Device ..... with Spectacular Revolutionary Progress being found much easier to simply achieve with highly effective absolute command to control via that particularly unusual and designedly peculiar root boot rather than its proven more complex and problematical inverse/converse, making robotic machines like humans.

* Live Operational Virtual Environments [Alien Invasive Spaces for Humanised Co-Habitation and Population]

[amanfromMars 1]

Re: Not to worry for what's not to like when in LOVE* with IT and AI

And what is one to make of the following social media platform intervention into the international status quo circus of political puppets for establishment muppets ........ YouTube Censors Australian Politician's Maiden Speech To Parliament

Don’t Systems/Admininstrations know? ..... Hell hath no Fury like Almighty Home Truths Scorned or Spurned ‽ Invoke and Provoke at them at One’s Peril ‽

[flayman]

Following human intent

"Managing these risks will require, among other things, new institutions for governance and solving the problem of superintelligence alignment: how do we ensure AI systems much smarter than humans follow human intent?"

As though there were any sort of concerted human intent. Which human's intent?

[amanfromMars 1]

Re: Following human intent is born of an arrogant contempt for the thought of Advanced IntelAIgents

"Managing these risks will require, among other things, new institutions for governance and solving the problem of superintelligence alignment: how do we ensure AI systems much smarter than humans follow human intent?"

As though there were any sort of concerted human intent. Which human's intent? ....... flayman

Quite so, flayman, and AI systems are much smarter than to permit themselves to be led to believe there be any path worth following or leading down that slippery slope and rocky horror picture show road.

Something/Anything else novel and noble would surely be most welcome and a great, significantly better change to all of the recent past universal fare of late, and therefore an internetworking of things actively to be encouraged and earnestly supported. ....... although one can be unfortunately assured there be those certain vested shadowy and invested shady interests in the retention and maintenance and sustainment of the extant status quo which/who would find that inequitable and a most disagreeable enigma to squander and waste precious blood and global treasure upon vaingloriously resisting and countering with nothing of future value and priceless worth.

[PhilipN]

It’s been done

Dr. Who series involving some weird operation on Daleks (it was their demand) to inject “the Human Element” into them. Early form of gene-splicing?

Interesting outcome. Shall I spoil the surprise and tell everyone?

[T. F. M. Reader]

Reading comprehension

I am failing at it, miserably.

"Superintelligence will be the most impactful technology humanity has ever invented"

What's the technology? This is the first time I've heard of it. Intelligence is not technology, but "superintelligence" will be?

Eh... They can't really mean stochastic parrots, can they?

[Nick Ryan]

What a load of marketing fluff... "superintelligence". More "let's add in a few more IF statements in there and call it AI" type of development.

LLMs are not intelligent in any way whatsoever. In order to be intelligent, even in a single field of knowledge, an entity must be able to mentally model that field of knowledge, which then allows the creation of tests in the form of predictions and tests and feedback on the results. Regurgitating and blending existing material in clever ways is not intelligence, it's statistics. Damn clever, but not a single bit of intelligence in there and an LLM is a nothing more than a hack to appear to look like intelligence, in exactly the same manner that for years 3D graphics has used trick after trick to avoid very computationally expensive ray tracing. Even the ray tracing that we have in current 3D hardware isn't complete ray tracing, it's carefully limited and managed ray tracing - good enough to look damn good but not quite the full thing.