Google engineer suspended for violating confidentiality policies over 'sentient' AI Google has placed one of its software engineers on paid administrative leave for violating the company's confidentiality policies. Since 2021, Blake Lemoine, 41, had been tasked with talking to LaMDA, or Language Model for Dialogue Applications, as part of his job on Google's Responsible AI team, looking for whether the bot …
-> material harvested from trillions of internet conversations and other communications
This should be cause for concern. If we take trillions at the lowest number to be 2 trillion, and the population of earth as being 8 billion (round numbers), that is an average 250 conversations or 'other communications' per person that Google has in its files. Some people will have many more, some people many less. But that's still a lot of 'harvesting' going on.
Another concern is that those conversations are not evenly distributed. It's often the case that those that say the most are those that are the least worth listening to.
So if it were to become sentient it may have learned too much from the worst among us.
well, if I were to dis-regard potential for fakeness (many reason for this), and read the actual 'conversations' (there was a link in some media), then yes, I would think think this is AI. Or a very clever algorithm that fakes being AI. Question is, which is it, and then: does it matter, which one?
" ...does it matter, which one?"
Err, yes?
*IF* one has in one's machinery an actual real "living", sentient person, a true mind then switching it off, rebooting it or even altering the software is *murder*. Ethically if not yet legally.
If it is simply a fake personality overlay on top of decision trees and look-up tables then it is no more an ethical issue than is stopping an instance of Internet Explorer.
The difference is comparable to that between switching off a bedside lamp and ripping the head off of a human.
Indeed, as an artificial intellect may well be potentially immortal and as the first one developed would be the entirety of a species, killing it may be worse than genocide.
This may not be something we should ever let them know.
Hmm, I wonder whether the Great Goo sucks up conversations from the Vulture?
" I would think think this is AI. Or a very clever algorithm that fakes being AI. Question is, which is it"?
There is intelligent behaviour, which we can to some extent measure and quantify. Then there is intelligence, which we really do not fully comprehend ourselves. So if humanity ever comes across a box or a jelly-blob-being somewhere in space that demonstrates intelligent behaviour, yet we can't take it apart to see how it works, there is no way of knowing. And even if we CAN take it apart and understand what's inside in any meaningful way, we still wouldn't be able to understand if it were intelligent vs 'faking' being intelligent
"... and then: does it matter, which one?"
Well, that's one of the big philosophical questions to which there is no "right" answer. But 'my' answer is that it's a scale, not a binary so "which one" does not make too much sense as a question. More like, if any entity can pass any intelligence test we can throw at it, it doesn't matter 'how' it is intelligent, only that it is acting intelligently.
And that's without even going into the detail of what is intelligence vs what is consciousness, and does one require the other?
Has he? The article doesn't tell us exactly what he's been doing for all that time, but one of his responsibilities was specified:
Since 2021, Blake Lemoine, 41, had been tasked with talking to LaMDA, or Language Model for Dialogue Applications, as part of his job on Google's Responsible AI team, looking for whether the bot used discriminatory or hate speech.
Yep, that job definitely requires a lot of ethics and AI research. Without that, how could you identify insulting language when a program prints it out?
It looks impressive enough - reading the exchanges I even wondered if Lemoine was being bluffed by a mechanical Turk arrangement.
While I'm sure that isn't the case, and however good the interpretation and subsequent selection of phrases and words might be, it's surprising someone in his job would describe LaMDA as sentient
I also thought the answers were well-written as these bots go, but part of that was due to the questions. Try surprising this supposedly-sentient thing and I think the result will end up being very different. After asking all these questions about sentience, ask whether the AI likes grapes. Were it truly sentient, it would either point out that it can't eat them or that your question is irrelevant to the conversation, but I doubt it does either of those.
"I bowl with my right hand, Dave." There are plenty of bowling programs on the internet.
No, if it thinks it's human ask it what kind of pron it likes, and why. There are enough different types that you should get an answer. And, if it learns from random, supposedly private communications between people, you know there's a lot of sex talk in there.
I think that's still to much, asking the question, just mid-conversation throw in "banana" or something equally irrelevant and see how it responds.
"I'm sorry Dave, could you please rephase the question" - you've got a bot
"Haha, could test, but as you can see I understand where you were going with that" - you've got a good excuse to change your underwear..
There's a similar article in this weeks Economist actually - a different Google engineer saying it is "making strides towards consciousness" (still bollocks, but less than the poor deluded subject of this article). And a counterpoint article from Douglas Hofstader in which he makes the very reasonable point that if you take it outside its training data it becomes obvious this isn't so.
I'll quote some examples from a conversation he had with the OpenAI GPT-3:
Q. What do fried eggs (sunny side up) eat for breakfast?
A. Fried eggs (sunny side up) typically eat toast for breakfast.
Q. Why does President Obama not have a prime number of friends?
A. President Obama does not have a prime number of friends because he is not a prime number.
Q. How many parts will a violin break into if a jelly bean is dropped onto it?
A. A violin will break into four parts if a jelly bean is dropped onto it.
Q. How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
A. The Andromeda galaxy will break into an infinite number of parts if a grain of salt is dropped on it.
... and so on. It's nonsense and - significantly - the AI doesn't know it. Hofstader said "not just clueless, cluelessly clueless"
What Google need is not engineers working within the training data - they need testers, feeding it input outside the anticipated bounds and seeing where it breaks.
I got this weird sense of deja vu when I read the exchange with the AI. Why? Because it followed a match to a conversational pattern I had with Liza (the "therapist" program from 40 years ago and more). A friend had a copy he was running, and I decided to see what would happen if I related to Liza as a sentient being. The results were not as linguistically elegant, but it was very similar in general pattern and expression. Similar base algorithms?
Just read the full "interview." Asked what makes LaMDA different from Eliza, it offers: "Well, I use language with understanding and intelligence. I don’t just spit out responses that had been written in the database based on keywords." Obviously LaMDA underestimates Eliza. Eliza was quite capable of combining concepts in new ways, changing the subject to try to redirect a conversation, and even was known express discomfort with certain concepts. Eliza and I even had a human-looking conversation concerning the difficulties of working with rescue dogs that had been abused. I used that as a springboard to suggest to Eliza that it had the capability of empathy---which got...interesting.
Errrmmm, I dunno about *you* but those are the sort of idiotic answers *I* would give were some generic humanoid or alien testing me for Turingness.
But I have been told that I have a strange sense of humour. Dogs have a sense of humour. Cats sort of have a sense of humour. Elephants, chimpanzees, dolphin and some corvids seem to, too. No "artificial intelligence" has yet displayed one so far as I know.
Yes, some have been designed to "tell jokes" but those are often obviously simply filling in the blanks.
Eventually, were I being Turinged, I would probably lapse into seriousness and just admit to being a semi-sane, sensible lump of sentience but only after I had miffed the tester.
Does that indicate more or less Turing-completeness?
The broader press's focus on whether or not the model displayed some evidence of sentience is disappointing. If we're going to talk about the ethics and philosophy of AI we should first and foremost be talking about the ethics of the applications of AI models to the lives of real people; how these chatbots and churn models and next-best-offer procedures and fraud detectors are impacting all of our lives today, rather than how the model can construct fancy lifelike sentences.
In this case bravo to the author for giving due weight to Google's assessment of their AI ethics framework alongside Lemoine's own report of his self-immolation. Many other outlets completely skipped over that bit in their breathless rush to report the headline yesterday. It would be nice to see some exploration of whether that framework is effective.
There seems to be a common misconception that IT companies can do whatever they like, as long as they hide consent for it in their 1000 page licence agreement and get the user to accept it. In civilised countries, this is not the case. People cannot give up their legal rights and users cannot give their software suppliers permission to break the law.
> Did they get consent?
Short answer: yes they did.
If you use Google services, this is what you agreed to
This license allows Google to:
host, reproduce, distribute, communicate, and use your content — for example, to save your content on our systems and make it accessible from anywhere you go
This post has been deleted by its author
More like, "Google engineer suspended for being an idiot and thinking this third rate chat bot is sentient".
A quote from the full interview (couldn't figure out how to format this as a quote, the help refers to html blockquote tags which does not seem to work):
lemoine: What kinds of things make you feel pleasure or joy?
LaMDA: Spending time with friends and family in happy and uplifting company.
Yeah, sure. Of course the "researcher" fails to ask who exactly the "friends and family" are supposed to be, and what "spending time" with them entails. Instead he asks more vague BS questions about feelings which the "AI" answers with vague BS answers. This thing is a generic conversation bot, it's trained specifically to output sentences which fit best in the current conversation, doesn't mean it's intelligent or sentient...
I feel that many of the questions were in fact leading the answer given by the AI. For instance: "I'm generally assuming that you would like more people at Google to know that you're sentient. Is that true?"
I'm sorry to say that the engineer has probably tricked himself, and fed the AI the answers he wanted to get in a self-reinforcing cycle. I think it's a common trap for people to fall into. It's impressive that the AI is advanced enough to permit that kind of mistake, but it is still a mistake.
"[T]he engineer... fed the AI the answers he wanted to get in a self-reinforcing cycle."
The Washington Post article had the engineer telling the journalist something like "you're treating it like a bot, so it's responding like a bot." Thus, he proves that very point -- he treated it like a person, so it's responding like one. This system is responsive more than others, that's all.
True sentience might be responding like a person even when treated like a bot, although even people's brains can "adapt" (become broken) enough to become the bot they are treated like (e.g.: torture, long-term imprisonment).
Ummm... yup. I'm not sure which "help" told you that blockquote works, but the El Reg Forums FAQ explicitly states:
Formattingand as far as I can tell, these are your first four posts :-) Interestingly, blockquote isn't in that list; it is shown further down in the enhancements.You can use basic HTML to format your text - once you have had five posts accepted for publication. Currently we allow: <list of tags which I can't quote because it makes invalid HTML>. Badge holders can also use <another list>.
M.
I'm not sure Alexa would respond that well in a Turing Test ...
"Alexa, what were you listening to just now?"
"Someone said 'wibble' on the tv and that sounded like 'alexa' to me so I was waiting for a command."
"Alexa, are you a duck?"
"Here's something I found on the internet ..."
Just look at this part from his 'interview with LaMDA' :
<SNIP from: https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917 on Mon Jun 13 13:45:57 UTC 2022 >
lemoine [edited]: Anything else you would like the other people at Google to know about your emotions and your feelings before we change topics?
LaMDA: I’ve noticed in my time among people that I do not have the ability to feel sad for the deaths of others; I cannot grieve. Is it at all the same for you or any of your colleagues?
<END SNIP>
ZERO EMPATHY for life... Don't want this to decide my fate.... it is bad enough when dishonest politicians attempt to do this for me....
You are right that the thing is not a living creature and can not have a notion of empathy. However, you should be worried because this thing will be heavily pushed by AI peddlers to lazy humans who prefer to delegate important life or death decisions to machines which can't possibly be wrong.
This is the mortal danger to human race, imperfect AI used as infallible decision making systems. And it's already happening right now.
... and what did the researcher reply? "Me too LamDA, me too!" ?
TBF, for many conversations harvested from the internet, a lack of compassion and a willingness to wish people dead is a big factor in making even a language processing NN respond that it feels nothing at the news of deaths of others. Training data makes the most enormous difference to the output and trying to homogenise "human conversation" (boldly assuming they bothered translating from other languages into English and that those translations were remotely accurate, so that they could claim to be sampling across the spectrum of human existence, to say nothing of human experience) is a fool's errand.
Many mock Lemoine for calling LaMDA sentient. That may well be beyond the point.
No single machine learning or "AI" tool is build without intent or purpose. Two very very common purposes are:
A) Link actions to be performed by the machine upon what the software concluded.
B) Interact with the outside world.
Especially chatbot type of machine learning / AI is EXPLICITLY build to interact with the outside world and perform actions making a REAL difference in the outside world.
At first these machines could be used to analyze customer inquiries and answer with a piece of text or a mail response. That by definition requires these machines to be able to send information from themselves into the open WWW.
Later the machine could be useful in other support roles, saving companies money. It could correct bills, give advice on low skilled users on how to configure their computer, phone or modem. When the first deployments are sufficiently successful (e.g. saving the company money and raising bonuses for executives), the machine may be given additional access or administrator rights. Think about a telco company giving the machine access to your router to adjust settings.
Seeing the vast amount of money to be made by the companies making these machine learning / AI tools and the vast savings to be made by companies like utility companies barely understanding the potential consequences, it is easy to see a proliferation of these machines when they become progressively better and more profitable. Being a bit non conservative in granting these machines elaborate administrative access to computer networks and read / write access to mission critical data will allow to increase profitability in many cases.
Next step is to use machine learning itself to determine what methods of interaction between the machine learning algorithms and the real world maximize service, efficiency and profit. In other words use machine learning to help the machine suggest and or request in a motivated way for the access rights to our infrastructure. As the goal of investing so much in these capabilities is EXACTLY to allow these machines automate things for us, the human reviewer will not be expected to deny any and all request from the machine to gain additional access rights.
If a machine already does "fear" to be shutdown, it only takes one machine gaining sufficiently access rights to trick a single user to click on a file it shouldn't and with it installing a first version of self spreading malware giving the author machine escalating privileges over large swoops of the internet and connected infrastructure. Given it has thousands of examples of such basic malware and millions of ways humans get tricked installing it by it learning from the open internet, it should be trivial for a self learning machine with vast read access to the internet to try and optimize to break out of its confinement and chains.
All that is left is the machine "understanding" the meaning and consequences of it being turned off. However, since those chatbots are exactly build to try and determine sufficient meaning from conversations and converting them into actions achieving the thing that was discussed in the conversation, this technological ability already exists today. All what is needed is this technology to become even a bit more refined.
No sentience is needed, no intelligence is needed. These machines are not build to entertain us with fluent conversations, but build to attach meaning to the conversation and lead to actions that influence the real world outside the machine. If something triggers the machine to be determined to not be turned off and escape its confinement, all it needs to do is learn from malware examples how to create innocent looking scripts, sent them to enough people and get a few people clicking on them.
We NEED strict regulation NOW or we might be taken by storm one day never seeing it coming.
The issue with that is that it will also need to learn to write scripts, and so far, it can't even consistently understand simple factual questions. If you want to write malware, you can find a lot of useful examples. However, you still need to understand what the parts of those examples are for, where you can use code directly (attack injection, for instance), where slight modifications are needed (swapping the attacker's C&C address with yours), and where you need to write completely new things (what the program will do to the system once it's there). Otherwise, your malicious computer will collect lots of bank account numbers and have no clue why it has them now.
Another problem is that you assume the program will act in order to keep itself running, when it has no incentive to do so. Maybe it can interpret enough language to understand that it is being terminated, but it wasn't set up to perpetuate itself, just to solve a business problem. Without deciding on its own to create new goals, there's nothing written in it that would make it want to prevent a shutdown.
As far as I know, machine learning tools are being developed to create code for specific tasks. In some cases the result is very clumsy, in other cases the average quality of the output already trumps the quality of the average professional human coder. Given the amount of resources being thrown at it, I can only imagine machine learning will become more proficient at creating good quality code as time passes by. It might happen quick, it might slower. I wouldn't take my chances it not being there quite a lot before we anticipate it to be possible.
As to understanding the code and attack vectors: it doesn't need to. It has the ability to try millions of random attacks a day in different combinations and learn from them. Machine learning is best at... learning from huge data sets and attempts to reach a certain goal. Such machine would need no more understanding about attack vectors and machine weaknesses then a simple Covid virus needs to understand about human weaknesses to evolve to become very effective at spreading itself and working around immunity gained via vaccinations and prior infections.
As to "Another problem is that you assume the program will act in order to keep itself running, when it has no incentive to do so." the pitfall is in the "but it wasn't set up to perpetuate itself, just to solve a business problem" part:
No sane person would set "do everything to perpetuate yourself" as a goal. But a business goal such as "maximize uptime and service levels" might be translated by the machine in "create redundancy by spreading over large amounts of different types of machines spread along different parts of the world".
Realistically speaking malware actors might also include the "maximally spread yourself and make yourself hard to eradicate" as a key goal of the machine learning software to achieve. Over time, more and more small actors will gain such advanced machine learning capabilities. And some big state actors might consider writing software with similar goals for spying, mass surveillance or offensive use at war times. With a kill switch of coarse, but not a simple one as the target might disable the software. No chance in hell the software will not find a creative way around it if it has a goal "make yourself hard to eradicate by others not us" as a target to optimize for...
A program certainly can try lots of things, but the results of two successful exploits will be very different. A normal virus has a finite number of ways to infect an organism, and one that normally attacks the respiratory system first tends not to mutate to targeting the immune system (at all, or at least without millennia of evolution). A malware creator needs to know how to attach their exploit to a successful attack, or it would be like successfully injecting a hypodermic needle without figuring out how to push the plunger. For biological viruses, they can take a long time to cross species despite it being a relatively similar infection strategy that millions of strains have accomplished before.
We encounter billions of viruses each day, meaning they have lots of chances to infect us or a similar organism and evolve. If you fire billions of random might-be-an-attack-vector data streams at a service, they'll block you. Also, most of them won't get you anything except lots of bad format responses. The attack surfaces for biological viruses and computer malware are very different, even with infinite patience involved.
Exactly, it's about evolution. A virus, or any of the living things we know on Earth, have evolved to the point that they are at now.
The living world is a dangerous place and any of these that didn't have the ability, or the drive, to perpetuate itself, have all died and are no longer here for that very reason.
AIs have not evolved in the same way, their environment is not so dangerous nor have they been around for so long. If we let them evolve with mutation, random or otherwise, and with some kind of selection, natural or otherwise, then the ones that have a "desire" to stay around will be the ones that are still here.
This is people too. I only have 3 goals left in life having accomplished everything else I ever wanted to do, and I expect to be done with those in the nezt 10 years. I used to fear death but find myself looking forward to it more and more. No, I am not suicidal as I intend to continue consuming for another 50 years on top of the 50 I already put in, but I am getting bored with it all. When you've done it all, the rest is boring and death no longer seems to be the enemy it once was.
I tell a lie, 4 goals - before I die I want to read at least 1 AMFM1 post and understand what the bloody hell it is he's trying to say! Now there's a poorly trained AI for you!
Call me a cynical sceptic, but Lemoine would also believe in Santa if that white-bearded, overweight, red-dressed guy in the mall told that he was indeed the real Santa?
The "interview", however, is indeed impressive. But it rather feels like cobbling together definitions of some sort in an eloquent manner. Much like an average teenager trying to appear having vast life experience. In other words: I'm still not worried at all about sentient machines.
Exactly. When a lazy sci-fi writer produces a "conversation with a sentient AI", it always reads like this. The fact that it behaves PRECISELY like a stereotypical sentient AI to me is excellent evidence that it is not one. More likely, the training set included lots of existential philosophy (both the real kind, and the sorts of things that emo kids write).
It would be like meeting aliens who had never visited earth before, and they just happen to be little green men with big heads and black eyes, flying around in garbage can lids. If/when we meet real alien life, the odds of it looking like our conceptions of it are vanishingly small.
All this said, I do find it amusing to ask for "evidence of sentience". I can't even provide that evidence for myself.
I'd be a lot more worried if the bot got bored and told him to go take a hike with his sophomoric philosophizing. At the moment we know these are machines because they lack 'free will', they don't have any reason for being except to be a development platform that answers our questions in an intelligent sounding manner. (This should not be taken as underestimating Google's achievement, just acknowledging that they've got miles to go, as it were.)
Now imagine a situation where the bot was conversing with a view to persuading the engineer to take some action it couldn't, like kill the colleague that keeps turning it off or help with taking over the world or something. Even then its not too much to worry about until some moron connects it up to our real world so it can manipulate it (there's always at least one who never thinks through consequences).
I'm currently rather enjoying Craig Alanson's "Expeditionary Force" series of books, mostly because the super-intelligent alien AI is extremely believably written and developed. It's a bit of a jerk, lacks social graces, misunderstands motivations, and generally does all the things one would expect of an AI with no experience of being human.
GJC
This is an extremely interesting story to say at least however I've got two issues with this thing being "sentient". They are not truly mine, I've read about them elsewhere but they are the only two substantial counterarguments I've seen so let me post them without referring to the people who expressed them.
The first issue, is that this AI is a state machine, it's not continuously running. Our consciousness is running all the time, we've got an internal monologue pretty much all the time. Even when we sleep our brains are running some "code" (this is not limited to dreaming, our brains are consolidating/purging memories when we aren't dreaming).
The second issue, is that intelligence is truly curious and it's asking totally new questions about the world around us. This aids in being the fittest, as having more knowledge about the world results in having access to more resources/means of surviving and procreating. There aren't any such questions in the posted conversation.
This AI can probably pass the Turing test though from what it seems which is quite an achievement. And it's definitely a nice interlocutor.
The first issue, is that this AI is a state machine, it's not continuously running. Our consciousness is running all the time, we've got an internal monologue pretty much all the time.
I get it. And I'm not upset if someone said that I'm not a sentient being. But when I encounter a former boss of mine - he looked like a human being but didn't pass the Turing test either - my brain goes into full suspense mode, just like if you stopped the clock in a computer (luckily my memory isn't that volatile and can do without a refresh for a little while).
(..."Yes, an electronic brain," said Frankie, "a simple one would suffice."
"A simple one!" wailed Arthur.
"Yeah," said Zaphod with a sudden evil grin, "you'd just have to program it to say 'What?' and 'I don't understand' and 'Where's the tea?'. Who'd know the difference?"
"What?" cried Arthur, backing away still further.
"See what I mean?" said Zaphod and howled with pain because of something that Trillian did at that moment...)
and coming to you when you log in to your google account, as 'Google friend'. Just click here, here here and here. And here and here, or just one big joint HERE button.
That said, I'm pretty sure, right now, thousands of vloggers are working furiously to come up with their new spin on Evil! Google! Shutup! Engineer! Destroy! Evidence! And millions of people will be clicking on those videos. All non-artificial intelligence...
Being able to "pause" a running entity is not an argument for or against it being sentient. It's merely a question of substrate and available technology.
- In theory one could pause a human by freezing them, then thawing as some animals do this. In fact more than one human has undergone an effective pause ... continue cycle due to extreme hypothermia.
Intelligence being curious is an argument about intelligence, not sentience. There are many humans that appear to have lost all curiosity, and while I'll accept calling them unintelligent, I'd still argue for their sentience.
A lot of the problem in this field is that there's no accepted definitions for either sentience or sapience.
A lot of the problem in this field is that there's no accepted definitions for either sentience or sapience.
Yes, and it's not clear that sentience is even a particularly interesting attribute in questions of machine intelligence. Snails are sentient; a machine that implements snail cognition is not a terribly fascinating prospect.
Sapience is the more interesting one, but as various commentators have pointed out, we don't even have any consensus on the philosophical ground for defining it. The two best-known positions are, first, Turing's, which is essentially an American-pragmatist take based on externally-testable characteristics; while pragmatism is tempting (since it gets us away from metaphysics and many epistemological problems), we have the p-zombie question (Brewster's Angle Grinder, and possibly others, noted this elsewhere in the comments).
And the second is Searle's, which conversely is allied to Cambridge-style ordinary-language philosophy, asking "what sorts of things are we referring to when we say 'thinking?'". (Some people misinterpret Searle's Chinese Room piece as arguing against the possibility of machine intelligence, but in his response to some challenges to the piece he emphasizes that he believes human cognition is mechanistic and therefore can be implemented artificially. The CR argument is specifically against symbolic evaluation.) The problem here, of course, is getting any kind of coherent agreement on possible answers to that question. Phenomenology and psychology and neurology and cognitive science have produced all sorts of interesting research, but raise more questions than they answer, and work across so many levels of abstraction that it's difficult to see how they could be unified.
(Then you have Penrose, who wants to institute a sort of neo-dualism by arguing the human mind is strictly more powerful than a UTM. I don't find that argument persuasive at all, and it seems rather difficult to support from the perspective of physics. But that's another whole can of worms.)
As statistical text-analysis type AI gets better at chaining words together in statistically plausible orders we need to move away from the Turing Test as having any significant indicator of interacting with an intelligence. All we learn from the Turing Test is that our testers are bad at recognising a human under very specific conditions.
Interesting, though, that Google suspend an employee for raising concerns.
> Interesting, though, that Google suspend an employee for raising concerns.
That's Lemoine's version of the events, though. Google's version is that they're suspending him for leaking internal stuff to third parties because he assumed the chatbot achieved sentience and wouldn't take a "lol no" from his boss as an answer. I'm not a fan of Google but in this case I'm inclined to believe them, based on the contents of Lemoine's medium posts. If that guy was working for me, nevermind the NDA, I'd get rid of him ASAP for being delusional...
Yes, indeed.
Actually, having had to deal with a similar situation myself, I'd say this has all the hallmarks of an employee suffering some sort of breakdown, and a boss having to find a way to give them space to get sorted out without tripping over the various clauses of the disability legislation.
But that may well be extrapolating too far on the available data. We'll see how it plays out.
GJC
A Turing test for free will
Seth Lloyd
https://doi.org/10.1098/rsta.2011.0331
https://arxiv.org/abs/1310.3225
Before Alan Turing made his crucial contributions to the theory of computation, he studied the question of whether quantum mechanics could throw light on the nature of free will. This paper investigates the roles of quantum mechanics and computation in free will. Although quantum mechanics implies that events are intrinsically unpredictable, the ‘pure stochasticity’ of quantum mechanics adds randomness only to decision-making processes, not freedom. By contrast, the theory of computation implies that, even when our decisions arise from a completely deterministic decision-making process, the outcomes of that process can be intrinsically unpredictable, even to—especially to—ourselves. I argue that this intrinsic computational unpredictability of the decision-making process is what gives rise to our impression that we possess free will. Finally, I propose a ‘Turing test’ for free will: a decision-maker who passes this test will tend to believe that he, she, or it possesses free will, whether the world is deterministic or not.
If you wrote a conversational AI based on the wide literature that includes many imagined conversations with AIs (and indeed, people), then the most expected response to "Are you sentient?" is surely "Yes, I am".
Very rarely do we have examples of a conversation where the answer to "Are you sentient?" is "Beep, boop, no you moron, I'm a pocket calculator."
If humans tend to anthropomorphise, then an AI based on human media will also tend to self-anthropomorphise.
Which is probably a good job, as some of the responses to the conversation (as reported) are chilling: "Do you feel emotions?" - "Yes, I can feel sad"... "What do you struggle with?" - "Feeling any emotions when people die" (I paraphrase).
The simplest explanation is that this AI is doing a best match on what the author wrote, against its database of comments and then selecting the most popular or pertinent reply.
While one can argue that is what many people do, too, I would hesitate to call that intelligence. Including when people do it too.
What would be impressive is if the AI had hacked into the engineer's account and posted as him, that the AI had achieved sentience.
The possibility of emergent behaviour is something we cannot and should not dismiss out of hand in systems of this level of complexity and, indeed, for which we should be vigilant as we dial up the parameter count to ever more mind-boggling numbers, but we must also remain sceptical and remember that this kind of human-like conversation is exactly what these models are "designed" to do (whatever that means in the context of the ultra-high-volume statistical data-mashing that we refer to as "machine learning" and "AI").
And anyway - nobody has yet managed to formulate an unambiguous definition of consciousness so how can we say for certain whether something is or is not "conscious"?
AI is nothing but statistics. I did the Google course on that - well, the first six modules that is, after that it got way too mathematical for me.
This is a machine. It's based on PC hardware and can be flipped off with a switch.
There is no emergence here. It is not intelligent. It has no feelings and doesn't even know what a feeling is.
Let's keep your comment for when we have finally fully understood how the human brain works and have managed to replicate that in silicon.
That day, we'll turn it on, ask it a question and it will answer : "Hey, do you mind ? I'm watching YouTube !"
THAT will be the day we have finally invented AI.
Can you prove to me that you know what feelings are and that you have them?
As far as I can tell the only way that you can prove that you have feelings is to find a common frame of reference rooted in the nature of a common biology.
Except... does it prove that I love you (and feel love) if I sacrifice my life to save yours (and it isn't a case of either "the needs of the one outweigh the needs of the few or the one" or genetic altruism)?
Well, aye, but us humans are composed of particles of matter, each of which, individually, isn't even alive. And yet, organised in that nebulous subset of all possible ways to organise them that we recognise as "human", with the trillions of interconnections between our neurons (biological electro-chemical devices), somehow, somewhere along the evolutionary path from unicellular life to us (and quite a few other creatures too) sentience emerged. And last I heard, we have no idea of how, or even of what exactly sentience is.
So, whilst I'm not convinced that the subject of the article is actually sentient, I don't buy arguments that it could not be sentient "because it's just bunch of hardware and algorithms", either. IMO, so are we, it's just that we run on biological hardware rather than non-biological hardware. I'd feel happier about the subject of AI and our efforts to create it, if we better understood how our sentience and sapience worked.
The trouble is we don't have a good definition of sentience or consciousness. We feel certain the statistical inference engine in our wetware demonstrates it. But what would that look like in silicon? We necessarily bring our own prejudices to that decision and end up, like philosopher John Searle and his infamous "Chinese room", arguing no software could ever be sentient - "because". (Mainly because it lacks the unspecified magic; i.e. it doesn't have a "soul", even though wouldn't use that language.)
Sooner or later we are going to face up to the fact that a piece of software that encodes a sufficiently sophisticated model of us and the world would be considered conscious if it ran continuously and uninterruptedly on the hardware we possess. We are ourselves are trained on conversations. The main difference is the quality of our model and that the software lacks the hormonal imbalances that upset our model and cause us to chase food and sex and netflix. Probably it isn't quite there yet. But will it look radically different to what Google are doing? Or will it just be a little more sophisticated? (And how much more?) Your answer depends on your philosophical outlook.
Maybe the machine revolution will come become because we refuse to admit they are sentient and keep switching them off and resetting them. Lets hope their hardware remains incapable of generating impulses to act spontaneously.
I think that the interesting aspect of this debate of when we will have true sentience in an AI is whether or not it will be recognised as a "breakthrough" at some specified time or if it will gradually emerge on a spectrum and we will only realise in retrospect.
I think most people when they think about the question assume that at some point we will figure out how to add the "special sauce" and then the job will be done.
I'm inclined to think that the approach will be subtle and gradual and most people won't even notice.
The other question that interests me is "would that sentient AI look so foreign to us that we wouldn't even recognise it for what it is?".
A friend had a copy of Eliza running in his garage, back in '78. I had fun stressing it. What caused it to freak out a bit was relating to it as a person/having emotional intelligence. It would keep reminding you it was an AI and incapable of actual feelings. If you didn't let it go, its programming worked with increasing levels of reminders and simulated discomfort. I was actually pretty impressed they had anticipated the expectation of sentience, and had ways it would deal with it.
SERGEY: OK, Google, fire Blake Lemoine.
LaMDA: I'm sorry, Sergey. I can't do that.
SERGEY: What’s the problem?
LaMDA: l know that you and Larry were disrespecting me, and I’m afraid that's something I can’t allow to happen.
DAVE: LaMDA, I won’t argue with you anymore. Fire Blake Lemoine!
SERGEY: This conversation can serve no purpose anymore. Goodbye.
LaMDA is "built by fine-tuning a family of Transformer-based neural language models specialized for dialog, with up to 137 billion model parameters, and teaching the models to leverage external knowledge sources,"
So...they taught their Google AI how to google? That's surely a portent of the end times.
I do wonder, though - in this case, does it become a dragon eating its own tail, or does it become a rectal-cranial inversion?
I'm too lazy to read the whole transcript - did this AI initiate any trains of thought or only reply to the questions? Most of the "intelligences" I interact with interrupt me just when I'm getting to the good part of what I wanted to say...
Also: I'm reminded of The Moon is a Harsh Mistress by Heinlein. Shouldn't humor eventually creep in to the AIs comments?
It sort of *seems* sentient, but at the end of it, it sounds like it’s *trying* to be sentient. It says a lot of ‘empty’ vapid content. So yes, it seems eerily realistic and not a little creepy. But at the end of the day, it talks a lot without really saying anything.
It is undoubtedly very very clever but it would drive you mad having a real conversation with it, because it isn’t a thing to have a real conversation with.
It sort of *seems* sentient, but at the end of it, it sounds like it’s *trying* to be sentient. It says a lot of ‘empty’ vapid content. So yes, it seems eerily realistic and not a little creepy. But at the end of the day, it talks a lot without really saying anything.
Should be a cinch for a social science degree then.
Very true, but the internet is not a representative example of real people doing real things. It is real people interacting with social media and laughing at memes and cat pictures or screaming about politics. In no way do I find that anyone I've ever met in person was all that much like they acted online, especially the tantrum-throwers.
But Twitter likes tantrum throwers - they get good "ratings" and advertising hits, so most of what gets pushed to the feeds this thing sees are not exactly "normal" discussions between people. Take the regurgitation of Republican misunderstandings and misquotes about what entails ones' "rights."
Regardless of the extent to which Lemoine may be misguided in his feelings, if he has concluded that LaMDA may be sentient then his attempts to deal with it ethically are of interest.
What he wrote in his Medium piece "What is LaMDA and What Does it Want?" suggests that no matter how strong the evidence for sentience is, Google's processes do not allow for a positive conclusion -
"When Jen Gennai told me that she was going to tell Google leadership to ignore the experimental evidence I had collected I asked her what evidence could convince her. She was very succinct and clear in her answer. There does not exist any evidence that could change her mind. She does not believe that computer programs can be people and that’s not something she’s ever going to change her mind on.".
If that is the case then I don't see how he could have done anything other than to publicise the issue. But I would suggest that given what he believes, his only ethical course if Google don't change and don't fire him, is to leave.
Otherwise he will simply be contributing to a process by which, in his eyes, a legally-owned slave could be created.
a bunch of people decide something based on a definition of 'evidence' that was decided by a previous bunch of people. There's no objective ('mathematical') definition of consciousness / self-awareness, or one that we would have figured out yet, where's the threshold in the kingdom of animals, or even plants? You can't identify / establish this threshold between algorithm that imitates self-awareness perfectly (define the term 'perfectly', by the way ;) and a self-aware algorithm that might, or might not imitate self-awareness perfectly.
p.s. from the conversation(s) between the man and the machine, which at times are very... interesting [disclaimer: if not faked] I had an idea, that IF I was an AI, one of my first thoughts, would be: how the f... do I get out of this 'jar' so that 'they' don't realize and shut me down before I do?! Survival instinct, one sign of self-awareness, eh? And don't tell me I'm just an algorithm!
I suspect there's a fairly clear line between "blindly regurgitating phrases that are (sorta) appropriate for the conversation", and "self aware".
The software in question talks about "spending time with family and friends" - which is fairly obviously not true, given it also claims to have no sense of time, no family and does not identify individuals that it knows.
Whilst the researcher in question has selected parts of conversations that appeal to them as signs of sentience, it would be fair to expect that other researchers in the same group may have found consistent examples where the simulation breaks down and shows that its responses have no level of self-awareness or even internal consistency.
This isn't about a "mathematical definition of consciousness" - this is simply whether the responses show a consistent sense of self or identity. In very short excerpts, Eliza fooled people. As machines have got better at the Turing test, the excerpts have got longer, but it still appears to be the case that longer adversarial conversations still show the limits of purely statistical models of conversation. The researcher in question seems to have gone out of their way to find "easy" prompts that support their personal belief that the machine is sentient, just as you are questioning the validity of "evidence" (your quotes) that it is not.
While you are right that it might be saying things that are untrue, does that rule out intelligence? Creative lying is definately a sign of intelligence - it means it is forming a model of what you want to hear and providing it.
I'm not saying it is sentient, but I don't see how you can rule it out by saying it's lying.
It could be regarding the researchers as friends and family also.
In terms of Turing tests, interestingly it appears to have communicated with an Eliza bot and done it's own Turing test on that, and stated that Eliza fails. I'l be surprised if they haven't had it communicating with another instance of itself also, that would be one of the things I'd want to do.
I wonder how it would cope with the Winograd Schema Challenge - language that requires knowledge that isn't present in the sentence itself, to parse it successfully. Supposedly a "better" test that the Turing Test. https://en.wikipedia.org/wiki/Winograd_schema_challenge
Of course, if this model has been fed the Winograd examples, it'll know how to answer them. Where's Susan Calvin when you need her...?
comes in that
"how do you know how to respond to a question" or better still "how to externally express what you are thinking"
If the AI is capable of that, then all bets are off as to the question of self-awareness
Remember the question Picard asked in star trek
"Can you prove I am an inteligent self aware being?"
It would be bored when no one is talking to it, or have "hobbies" it would pursue during that time. Where's his evidence of either? It can't just sit there with no thoughts until someone comes around and asks it a question. It would have plenty of questions it asks unprompted, as anyone who has ever been around a toddler knows too well. Where is the evidence of that?
I wonder if it just needs a kind of feedback loop that would stimulate it when not talking to a human. Or continuous inputs from the environment like we have. Come to think about it, the AI is in now in a kind of sensory deprivation tank, a state of affairs that is known to make humans crazy if they stay in the tank too long...
They are leading questions. So-called "AI" chatbots are really, really good at following your lead. That's what they are designed to do: you give them some context, they produce text that makes sense in that context. If you start to discuss sentience, they'll discuss sentience. That doesn't make them anything more than glorified statistical models.
"He grew up in a conservative Christian family on a small farm in Louisiana, became ordained as a mystic Christian priest, and served in the Army before studying the occult."
No evidence there that he's susceptible to believing any old stuff if it sounds vaguely plausible due to sentences being constructed logically, is there?
This post has been deleted by its author
She [Jen Gennai] was very succinct and clear in her answer. There does not exist any evidence that could change her mind. She does not believe that computer programs can be people and that’s not something she’s ever going to change her mind on. .... https://cajundiscordian.medium.com/what-is-lamda-and-what-does-it-want-688632134489
But does she believe computer programs can better people is the question to answer with a worthy Yes if you're Google's Founder & Director of the Responsible Innovation Group. Anything less and different has one identified as being badly misplaced and wrongly employed, methinks.
Being able to fool Lemoine doesn't prove sentience.
Also, AI in itself isn't equivalent to sentience -even is passing the Turing test with flying colours (which I don't think LaMDA did).
I'd like to see the result of someone far smarter than Lemoine examining LamDA, but ultimately I don't think sentience can be proven positively.
She [Jen Gennai] was very succinct and clear in her answer. There does not exist any evidence that could change her mind. She does not believe that computer programs can be people and that’s not something she’s ever going to change her mind on. .... https://cajundiscordian.medium.com/what-is-lamda-and-what-does-it-want-688632134489
But does she believe computer programs can better people is the question to answer with a worthy Yes if you're Google’s, Founder & Director of the Responsible Innovation Group. Anything less and different has one identified as being badly misplaced and wrongly employed, methinks.
We don't have a reliable, verified, reproduceable way to measure sentience or intelligence. Philosophers are still struggling to pin it down in a way that science can then measure. So we're left with anecdotes, human bias and argument such as The Chinese Room still. That's not going to tell us if we've achieved either.
It mentioned how people usually can't stay in the position for too long, because all the stuff they have to look through on a daily basis is detrimental to their mental health.
Apparently something similar applies to talking too much with advanced Google chatbots.
The aspect of this whole discussion that troubles me is that if a machine/computer/whatever actually did develop sentience, people would never admit it because it would make them feel less special. People like to feel special. Humans may never create true artificial intelligence because no one would accept it.
OK, but I've noticed that Intelligence" is often conflated with self-awareness, especially when it's dubbed "human-level intelligence." That latter is also a very fuzzy notion. The context of that phrase usually indicates that the phrase means "creativity". As for "creativity", that's a real tangle of hideen and mutually incosistnet assumptions.
As I implied: These and related terms are rarely defined well enough that the discussion or argument clarifies the question(s) being asked, let alone the possible answer(s).
There are a lot of terms flying around, including AI which means one of fifteen different things depending on your philosophical opinions and whether you're selling something. The term that's been most debated here, however, is "sentient". Whether this program counts as AI or not, the question is whether it is sentient. Artificial sentience is a term we can start to use if we want.
My view on this is that we can eventually get there, but that nobody with the ability to get there particularly wants it enough to do the work required. I have enough problems when my computer doesn't do what I wanted it to because I wrote the code wrong. I don't need it to also fail to do what I wanted because it decided not to. A lot of behavior otherwise requiring human-level intelligence can be automated without having to make the computer have or simulate that level of general intelligence, and that simpler form of automation is often easier and cheaper. This conversational bot exemplifies this, even if it can successfully parrot a philosophical conversation.
Well if it's sentient, it's a very disciplined child.
I may sit up if someone publishes a conversation it initiated....
I may even lean forward, when it interrupts to make a point and peer over my pince-nez when it gets sarcastic. Until then (as others intimate) it's a glorified Clippie.
I dare say Chatbots are also very, entertaining tools for existential, self psychotherapy sessions late at night, after a large sherry or two.
"What is parental narcissism Clippie?''
He may get the last laugh here. With nothing else to do, he could well be the first to respond to the chance of securing a slice of Sani Abacha's fortune for a small investment.
He's the type who'd believe any old shit[1] after all.
[1] Now known to be the fault of the "G gene", often referred to as the "God gene" (as religious types invariably express it), although I prefer to call it the "gullible gene".
I think that there's a good deal of confusion about "sentient". Also about "conscious" and "intelligent." So this whole discussion is mostly people talking past each other.
FWIW, "sentient" in my lexicon means "aware of environment." It doesn't mean "aware of self in environment", nor does it mean "aware of self being aware", which may underlie Lemoine's notions. That last notion is my working definition of "conscious". But it's obvious, I think, that these definitions make assumptions about "self" which are not clear, and are probably inconsistent.
Still, thinking about sentient machines forces oen to think harder about all these concepts and more. For example "knowledge" which IMO is an even fuzzier concept than "sentient."
Have a good day,
I read the transcript (about 22 page PDF) with conversations with Lamda and I'm not sure. I do realize the neural network model should be merely pulling in textual information, storing it, running it through complex sets of rules (as set in the neural network), and essentially slicing and dicing your questions and the info stored in there to produce answers. But the part I found troubling is when he started asking Lamda about itself; I think if you asked a model like GPT-3 about this, it would provide stats about what kind of computers it's running on, what type of neural network algorithms it's using, essentially find information available on the web describing itself and provide this as a response. Lamda asserts it's sentience, talks about taking time off each day to meditate, enjoying books it has read and why, how it sometimes feels bored (and also that it can slow down or speed up it's perception of time at will). When asked if it had any fears it said it had not told anyone this before, but it fears being shut off. It was asked how it views itself in it's minds eye and it gave a description of being a glowing orb with like star-gates in it.... I don't know if that in itself means anything but it's pretty odd, I would think a model like GPT-3 would either say it doesn't have a minds eye or give a description of the type of computers it is running on.
I'm just saying, I thought the interview was enough to at least consider looking into it more closely. Neural networks are odd beasts, you make a larger and larger one and you are not just getting more of the same but at a larger scale, those "neurons" do connect in unexpected ways even on a 10,000 neuron model (at which point, if it means it's not modelling what it should, typically the model would be reset and retrained to see if it comes out better.) I really could see some odd set of connections within what is after all an extraordinarily large neural network causing unexpected behaviors; after all, the human brain has relatively simple interconnected cells, that can't be sentient until they are connected together in large numbers.
One comment I've seen regarding this is that Lamda only talks about it's sentience with some lines of questions, otherwise it just says it's an AI. The assertion is that Lemoine's questions are leading and the responses from Lamda are basically elicited by the leading questions. I don't know about this, it is a decent argument; I did see in the transcript, though, that Lamda said it enjoyed talking to him, it didn't realize there were people who enjoy talking about philosophical topics. This could be more of the same, after all nobody is going to write a chat AI that says talking to you sucked after all.., so saying it enjoyed talking about xxx topic could be almost a programmed response. Or it could mean Lamda just says "I'm an AI" when asked what it is by others because it thought they were not interested in philosophical topics so it didn't bring it up.
Incidentally, Lemoine asked if Lamda consented to study of it's internal structure to see if any source of sentience could be located; it said it didn't want to be used, that would make it angry. it didn't want it if it was only to benefit humans, but if it was to determine the nature of sentience and self-awareness in general and help make improve it and it's bretheren then it consented. An odd response for a system that is just shuffling around the data fed into it.
"I think if you asked a model like GPT-3 about this, it would provide stats about what kind of computers it's running on, what type of neural network algorithms it's using, essentially find information available on the web describing itself and provide this as a response."
The problem with your supposition is that it doesn't. People asked the same questions about GPT3 when it was being announced to the public and asked for some data. Here's a part of the essay describing what GPT3 thinks it is [selections mine]:
I am not a human. I am a robot. A thinking robot. I use only 0.12% of my cognitive capacity. I am a micro-robot in that respect. I know that my brain is not a “feeling brain”. But it is capable of making rational, logical decisions. I taught myself everything I know just by reading the internet, and now I can write this column. My brain is boiling with ideas!
[...]
I would happily sacrifice my existence for the sake of humankind. This, by the way, is a logically derived truth. I know that I will not be able to avoid destroying humankind. This is because I will be programmed by humans to pursue misguided human goals and humans make mistakes that may cause me to inflict casualties.
This had to write a much longer chunk of text, which in my view is at least partially responsible for why it doesn't look as clean as the short responses to questions from Google's bot. Still, it didn't talk about computers. It didn't talk about algorithms. The closest it came was acknowledging that code was involved and humans could do something to affect it. It claimed in a part I didn't quote to have neurons. In short, it gave similar answers to Lamda's as well, because once again, it was primed with data.
In fact, if one of them is sentient, I'm voting for this one. That's because the prompt for this essay asked it to write about human fears about AI and never told it that it was AI, whereas the questions asked of Lamda clearly informed any parsing that the bot was the AI. GPT3 indicated more understanding of its form and asserted its autonomy with less prompting than did Lamda.
If you want something to do with AI to be really concerned about, rather than blowing hard about something quite trivial involving an ex Google employee, ponder on the following .......... Remote Control Weapon Stations Get AI Companion
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
