Remind me again...
Why anyone with half a brain or better thinks this imbecilic software is a good idea?
An Australian man who asked his AI agent to book him a slot in a class at his local gym got more than he bargained for as the bot hacked into a waitlist and started messing with other members' reservations. Australian broadcaster ABC identified the gym-goer only as “Andrew.” The report says Andrew was using the OpenClaw agent …
"
Given a task, they’re willing to do whatever it takes to accomplish it
"
Yes, but we are all told that winning is everything by the hyper competitive chaps that are at the top of society (and it is chaps for the most part).
Surely you dont want to be a loser - do you?
> Who would the courts find should be held accountable for breaking the law, if it came to that?
I guess nobody in this case; at no point did somebody commit the illegal act while intending to commit the illegal act. It'd be more similar to something like a mechanical failure, where the operator of a device wanted to do one thing but the device failed and did another. That is, assuming the court genuinely believed that the acts the AI bot took were unintended by the human in control. If he knew or should have known the agent would commit an illegal act, as if things like this become widespread, that'd be different. But then you'd have to wonder about the AI companies, since "... and don't do anything illegal" is usually implicit when you're using a company to perform a service...
I don't think courts will buy that final argument about expecting a service not to break the law, at least not once the service is no longer so novel. Asking an accountant employed by some big company to falsify tax records on your behalf doesn't absolve you of responsibility if they follow through with it [Edited for better analogy]. The company will not be in court because most criminal laws don't apply to companies. Your agent might be in court as well as you, but if the agent is not human then it'll be you only.
In this case it would hinge on what exactly he'd asked the AI to do and how he'd phrased it. "Is there a way to get further up the waiting list" is a fairly benign thing you might ask a receptionist (before adding ".. I'm willing to pay, it's very urgent" and smiling sweetly). But there are much more reckless ways to ask the same thing that might be seen to invite an AI that's known for breaking the rules to do so.
But yea, intent. The courts have hundreds of years experience wrestling with this concept so I'm sure they'll manage in these new cases, but it'll be interesting.
I expect the general approach will be to let consumer demand punish the tech companies for their AI breaking the law, sadly. Thus "our AI avoids breaking the law 89% of the time when asked to do so" becomes a marketing phrase, or similar ( https://www.theregister.com/ai-and-ml/2026/08/10/claude-code-puts-auto-mode-in-the-drivers-seat/5285326 ).
> I don't think courts will buy that final argument about expecting a service not to break the law
The final argument was that you'd have to seriously consider liability for the AI companies if they don't treat requests as by default constrained by the law.
I don't think courts would have a problem with that.
It should be considered but I don't think a criminal court would get much further than "this is a matter for legislators". Today they do not, for example, hold AWS responsible if a new user uses EC2 to commit hacking offences. I don't think there's any legal basis for that (in any jurisdiction I'm familiar with). If a user continues to use EC2 to commit criminal offences, the company knows about it and does nothing then the directors may ultimately become criminally responsible for aiding the user. That's a bit different (and extremely rarely enforced in practice). But there isn't a 'by default it shouldn't be possible to use the service to break the law' rule.
In any case the AI companies will say, with good reason, that they are doing what they can to constrain their service to abide by the law by default. Because of concepts like 'GDP' and 'light touch regulation' and 'lobbying' that will no doubt be enough for legislators for a good 15 years at least, judging by how long it's taken to get action against big tech companies in other areas where they facilitate and profit from rampant law breaking. (Not to mention that they're already shrugging about the same AI companies trampling all over long-established copyright norms).
"The final argument was that you'd have to seriously consider liability for the AI companies if they don't treat requests as by default constrained by the law."
If these AI agents don't know right from wrong, that's a huge societal problem. Parents often have to tell their kids (or should) that they can't have something when they want it. If a feature film is sold out, the proper thing is to look for the next showing and get tickets for that. I've been there and thought nothing of trying to get tickets for a different show or make plans to see the film on another night.
As far as paying somebody to get their place (tickets), I have no issue with ticket touts for concerts. I've thought of a different way to manage that and it doesn't need government resources. For a gym class, I could show up and wave a $20 bill (£20 note) to exchange with somebody and I'd probably be in or whatever premium would fit somebody's threshold.
You touch on the most important issue with AI:
Most people have been brought up with the normal societal constraints and norms, AI has not.
From that perspective you can't rate AI higher than a 5 year old or at best a rebellious teenager, but with a far better memory and a wider reach for causing absolute havoc. It's like 'bring your kid to work' but work is an early nuclear reactor on which the kid is let loose. And the results are already showing.
"Asking an accountant employed by some big company to falsify tax records on your behalf doesn't absolve you of responsibility if they follow through with it [Edited for better analogy]."
I don't know what your original analogy was, but I don't think this one matches the situation being discussed. The difference is the clarity of intent. If you ask for an LLM to find and exploit a vulnerability for you or an accountant to falsify tax records, you make your knowledge and acceptance of the illegal act very clear. This is not as clear because the user did not ask for a vulnerability to be found, let alone used, and tried to reverse the action. Using your analogy, it's as if I asked an accountant to find the way to pay the least tax on my income (thinking, but not stating, that they'd be looking for tax rules that apply to me) and they decided that false records was the best way of obtaining that goal. In that case, while tax law in a lot of countries would still hold me responsible and say it was my responsibility to review all the tax information I submitted, the situation would likely be considered when deciding on my punishment and I could likely sue that accountant for having caused that damage.
"The difference is the clarity of intent. If you ask for an LLM to find and exploit a vulnerability for you or an accountant to falsify tax records, you make your knowledge and acceptance of the illegal act very clear. "
That has me thinking that government should worry about tax law. A LLM that specializes in tax law and can winkle out even the most obscure deductions legally might drive the tax collectors mad. To date, the rich, people and corporations, have hired knowledgeable accountants that have been wise enough to charge for that knowledge. If I could input my circumstances to a trusted machine (work with me here) and have it give me a legal way to pay little to no taxes and tell me how to keep on a path that keeps the man out of my pockets, that AI would be oversubscribed.
There so much legalese to parse and so many permutations that it would a sophisticated program to work out whether getting a certain raise might lose you money and you should ask for another $100 which might let you do something to not incur any more taxes or at what point the next bracket takes more than your raise gives you. It can be done manually, but likely too much work that you are better off just plugging along and not bothering. Most people are happy about any sort of raise and don't see how it might be bad.
You're right.
Imagine that poor people would suddenly be able to avail themselves of all the tax loopholes! The horror! The world would end!
Yes, I'm only mildly sarcastic, I think it could be worth it. If we all paid a fixed amount of tax on actual income (read: all of it added up, wherever it came from) there would not be an issue at all. The stupid bracketing where a raise would land you in a tax bracket where you would not be better off is clearly inspired by the whole plutonomy investment idea (which, by the way, is very much in evidence now, thank you Citigroup).
"Who would be on the hook? The *ACCOUNTANT*, not the LLM."
Yes, but the person who's taxes those are will be paying lawyers to make sure it's all on the accountant.
Hallucinations would be bad for the rate payer, but real, legal deductions could be bad for the man. Making up your own deductions, getting made up stuff from AI or an accountant that doesn't know their beans are all non-optimal.
That isn't how tax brackets work.
To take a simplistic example, say the tax brackets are as follows:
£0-£10,000 - no tax
£10,000.01 - £50,000 - standard tax at 25%
£50,000.01 upwards - higher tax at 50%
If you earn £10,000 you pay no tax, you take home £10,000
If you earn £50,000, you pay no tax on the first £10,000, and 25% tax on the following £40,000, taking home £40,000
If you earn £51,000, you don't get taxed at 50% on it, you get taxed at 50% on the top £1,000, so you still take home £40,500
Admittedly, it's a bit more complicated when you take into account NI contributions, and means-tested benefits (because low-paying jobs don't pay enough to survive, that's another topic, because they should), but in a nutshell, if you earn more money, you don't end up taking home less due to taxation. This is a lie put about by people who don't want to pay even higher taxes on very excessive earnings. For example, we don't have a 50% tax rate, and we should, and we don't have even higher rates of taxation for people earning over £100,000 or over £1,000,000 or over £10,000,000, and whilst we have poverty and underfunded public services, they really should ratchet up at levels that are far above those that are reasonably needed to live, and live well. Roughly put, I'd advocate for raising the current thresholds, so the lower threshold is at a level where people can barely survive, the higher level is at a level of moderate comfort, and extra levels should be at roughly "can afford two luxury holidays a year" (50%), "can own a yacht" (75%), and "can own a super-yacht" (90%) levels.
I don't think they have to be that worried. Most of the obscure deductions are obscure because they only apply to very specific people or corporations. They earn little interest from the general public who can't use them but do get attention from those they benefit who do their best to keep it from being removed.
There are a few things that the average person can use to avoid some of the tax, but they tend to be easier to find either with a cheaper accountant or research that doesn't require diving into the law. I'm not sure the expensive accountants could help much more than that for a typical worker, whereas when they're faced with a corporation who does a hundred things and can find a way to define one of them as a mixed agricultural and industrial activity that qualifies for two deductions simultaneously, they can do much more for that client.
"There are a few things that the average person can use to avoid some of the tax, but they tend to be easier to find either with a cheaper accountant or research that doesn't require diving into the law."
Plenty of people don't take deductions that they are legally entitled to take. On an individual basis, that may not come to much. In aggregate, it could be quite a lot for the taxing agency. How easy it is to find and apply those deductions is key. Way back when Turbo Tax came on a CD and it's interview process wasn't sharing one's information with their "partners", I found several things I didn't know about. I could also run scenarios for only taking the standard deduction or itemizing (this is US taxes). I could also use the program to run things during the year to estimate the next year's taxes. Having chapter and verse to back up a deduction is important so diving into the law isn't a bad thing if you fall into a special classification for something. Chances are good that the tax agency staff won't know and take you to task.
I don't think this one matches the situation being discussed
The purpose of the analogy wasn't to exactly match the AI situation, it was to refute the idea that you can somehow blindly go about assuming a company providing an agent service won't do anything illegal on your behalf. Of course they shouldn't usually, but if they do and you've asked for it then you're responsible. (My original analogy did make the illegal request a bit more ambiguous but that wasn't the point, the rest of my post addressed intent).
I think history demonstrates that the "you asked for it" is pivotal and easily argued. A lot of people have successfully used the excuse that they did not ask for it, either because they actually didn't or because they were careful about how they did so nobody could prove it. It usually doesn't help their PR, but it does seem to be an effective way of avoiding fines or prison if you can back it up.
That makes combing through the prompt in the case of LLMs an important and subjective exercise. People would have to argue the intent of "Is there a way of moving up on the wait list" and whether that's sufficient to indicate malicious intent. They'd also have to argue about whether the request to undo the removal of the first guy indicates that exploiting a vulnerability was not the intent.
A lot of people have successfully used the excuse that they did not ask for it, either because they actually didn't or because they were careful about how they did so nobody could prove it.
The reverse is true too. Every day a lot of people are given long sentences for things they did not actually intend, but were not careful enough to avoid. Every manslaughter, every drunk or dangerous driving offence that leads to death or serious injury, every robbery or burglary that goes wrong, every gun that goes off on a movie set and takes out a couple of crew. (Notwithstanding that some of these are lesser crimes in their own right. The principle does not require that).
The sentences are less than with malicious intent, and a consequence of that is that if the effects are minor (such as one person being inconvenienced by their gym slot being cancelled and then presumably reinstated) then prosecution is unlikely, and it'll also be easier though not inevitable to argue you weren't being reckless if you immediately make best efforts to mitigate the harm.
But it isn't the case that you can simply argue "I trusted this reputable AI company's product wouldn't break the law for me" any more than you can blindly trust using your car, your wine, your (lawfully held) gun, or your tax agent, without proper care won't lead to harm.
"Asking an accountant employed by some big company to falsify tax records on your behalf doesn't absolve you of responsibility"
Sure, but if you ask an accountant to see if there is any way you can reduce your tax bill, and they falsify your tax records; that absolves you of criminal responsibility. The accountant is criminally responsible, but you still have to pay the tax.
It's a genuinely interesting question.
If a piece of software was deterministic and you instructed it to do something illegal, and it did it, I'm pretty sure you'd be liable, just as if you hit someone around the head with a frying pan, Bottom style, you'd be liable for it, even though it's the frying pan that whacked them.
If the software is not deterministic, and it does something you didn't ask it to, and that was illegal, I'd expect the maker of that software to be liable. If I sold you a robot that was ostensibly designed to make cups of tea for you, but then I'd designed it to also get up in the middle of the night and rob banks without your knowledge, Wallace and Gromit style, you'd be pretty quick to hold me liable for it.
So what about "AI", the magic bullet that everybody is firing into the air with the expectation that it will hit what they want, and no regard for where it's actually going to come down? It's arguably non-deterministic. Sam Altman hasn't programmed it specifically to go and commit illegal acts, but then it might do that anyway, if you word your instructions to it wrongly. I'd suggest that this is down to magical thinking, because "AI" isn't magical, and it certainly isn't intelligent. I'd argue that the liability lies with those who have sold it as having the capability of intelligence. It's every bit as dangerous as firing a kalshnikov into the air, because the results are just as unpredictable. "Agentic AI" has the illusion of usefulness with all the safety features of a bin fire.
I have been re-watching the Bottom episodes... upvote from me, too.
Watched the fortune-teller episode last night ("oh, while you're here, would you like to kick Eddie in the bollocks?" "Thank you very much. Don't mind if I do.") and also Guest House Paradiso ("I was going to have a little blue plaque made that said 'Gina Carbonara Stayed Here'!" "Well, now you'll have to get a slightly bigger one made that says, 'Gina Carbonara Stayed Here And Thought It Was Shit'...")
Ah. RIP Rik.
Rewatching all the Bottom material a few years back I concluded the Live shows were the pinnacle. Just amazing writing & creativity, confident over-the-top performances, so many in-jokes, and a very appreciate audience egging them both on. The kind of thing you watch and have to pause frequently because you just can't see through our own tears of laughter (though the original series had quite a bit of that too). Well worth a watch if you only caught the TV series.
https://www.youtube.com/watch?v=G_X_7TrcHGM
One of my life's regrets is that, although I was lucky enough to briefly meet and chat with Ade, Rik had already left us by this point, so I'll never have the opportunity to meet him, even though he was known sometimes to be a bit of a tit; Ade, on the other hand was, and I'm sure still is, an absolute gent.
They can pay the lawyers to avoid consequences.
Wherever the patsy is strikes me as more accurate, and that will be the users who have not read the Terms all the way, including the bit in a 6 point light gray font on white at the very end. Or those who use Microsoft Word?
If you ask your AI to "do whatever it can" to make something happen and then it does something illegal on your behalf, and you are surprised and make sure to tell it to never do that in the future that's one thing. But if you continue to use it to do that illegal thing knowing it is illegal, you don't have any "but it was the AI who did it, hold it or Sam Altman responsible" dodge.
Intent (over simplification):
I meant to kill him = murder
I didn't mean to kill him = manslaughter
Either way you're in a lot of trouble.
But this is weird as others have said, the tool is deterministic and may choose to fulful the command by unpredictable means.
The supplier has clearly stated 'this tool is unpredictable' but marketed it anyway. You have chosen to use the unpredictable tool, which has acted unpredictably and caused harm.
Who is to blame, I'd suggest there is a joint liability. The manufacturer has resleased a tool with capabilities way beyond requirements that act unpredictably with those capabilities; and has not adequately put controls in place to manage those capabilities. The user has used the tool in full knowledge of its unpredictability (because the manufacturer has told you) and it has caused harm.
Generally before you get access to a potentially dangerous tool or object you have to do some form of training to prove that you don't present a risk with it (cars, trucks, weapons, root), but with AI the powers that be seem to think it doesn't matter because it's a only a screen.
Yes but that's for crimes of violence where there is no way to undo it. For other crimes intent often determines whether you will be prosecuted or not, even if doing something without intent is technically against the law.
We're talking something incredibly minor here - I really doubt that you'd be prosecuted for taking advantage of lax website security to delete people ahead of you in line. The cops would tell whoever wants to arrest you "complain to the gym" and if it was the gym "just cancel their membership and ban them".
Exploiting a vulnerability on someone else’s computer is a violation of the UK's Computer Misuse Act (1990), and as AI is just a tool, the person using the tool is completely liable.
It is the users prerogative to sue the AI company if they feel they can prove the tool did not act in the way they expected, despite the AI company admitting it is unpredictable shit.
"Yes it is technically a violation. Do you think they will prosecute anyone for "hacking" a computer to cut ahead in line for a fitness class? Or that a jury would convict someone of a felony for that even if it charges were brought?"
They might. It's a novel test case for using AI to hack another system. While a spot in an exercise class is not murder, it demonstrates somebody's use of technology to get something they weren't entitled to. Next time it could be a bank account or fooling the bank's computer into putting your insufficient balance account into one with higher interest rates and then not penalize you for an early withdrawal.
I'd not cry if Ticketmaster got AI'd into selling all the tickets to a Taylor Swift concert to one person with a bulk discount. Whether TM had a robust enough security system to prevent that or not doesn't wipe out the intent.
In the case of a gym class, I don't see that as rising to the level of a felony, but certainly a misdemeanor.
"I really doubt that you'd be prosecuted for taking advantage of lax website security to delete people ahead of you in line."
It may also depend on how the security was circumvented. It's no different than hiring somebody with the skills to root the machine at the gym and make the switch. You could say that you had no idea that they were going to exploit a vuln, but the intent was to vacate a spot on the roster that you could grab when the class was shown as full.
In both cases, it's not that straightforward, unfortunately. It boils down to more than just agency, but also what the various parties involved could reasonably have known and expected in the situation. If the user could be expected to have known that the product was likely, or even only somewhat likely, to interpret their request in a manner inconsistent with their own legal obligations, and they asked it to act on their behalf anyway, without bothering to check its intended plan of action first, then they would likely be found negligent, opening them to civil or criminal liability for the agent's actions on their behalf. However - with all the marketing fluff and boosterism around AI at the moment, I think most would conclude that it's quite reasonable for the average person to conclude that a request such as the one in this case would be reasonable to make, in the expectation that the agent wouldn't attempt to break any laws in the process of fulfilling the request. In your example, this would be similar to the driver of a so-called self-driving car that was billed as having 'Full Self-Drive' when in 'Autopilot' mode, expecting that the vehicle will generally attempt to abide by traffic laws and not, say, move into the oncoming lane in an effort to overtake a whole column of traffic.
Where it gets stickier is negligence on the part of the company. They DO put safeguards in place to attempt to catch and prevent these kinds of actions - so many, in the case of Claude, that it can actively impair the useability of the software at times or in certain domains. The question, then, is twofold - did they go far enough with their safeguards, addressing all the possible failure modes they could reasonably anticipate; and were they clear, honest and upfront about the limitations of their product and its safeguards with the customer, in particular not warrantying or guaranteeing infallibility or misleading the customer as to the reliability or safety of the product. Clear and prominent disclaimers would be a core expectation here. The extent to which these conditions were met would, I think, be a long and drawn out matter for the courts, in precisely the same way it has been with self-driving cars. These companies have deep pockets, barnfuls of lawyers, and a lack of clear precedent in their specific domain niche. I suspect that their EBITDA is generally a far greater threat to them currently than any legal issues, and they're going to act according to those incentives unless new legislation (e.g. the EU AI Act, which they're already reluctantly responding to} comes into play forcing them to pay attention.
My guess is that the transcript would get brought in and lawyers for the user and the LLM company would get to fight about the particulars. Probably the user is going to end up getting the blame most of the time because they issued the command, even if they didn't know what it would do, but mostly because they have less money to spend on lawyers. Intent would be a big part of it, but if they didn't find something they could argue was sufficient intent, then they would have dueling negligence charges about whether the user should have been more careful with their prompts or the LLM should have been more careful with their guardrails.
This is very much akin to executives and politicians using legal advice as cover for white-collar crime. "I honestly thought it was legal to do [thing]!" isn't the world's worst defense, until you realize that it's also likely not illegal for the lawyer to deliver a bad opinion, and in the end, nobody's accountable at all, and the best justice can hope for is giving up illicit profits.
This situation's pretty similar. The user didn't tell their AI to break laws. And it wouldn't surprise me for one second that even if you told your AI "don't break the law", it did so anyway, because they are so lazy and goal focused, they routinely ignore meta-requests trying to regulate their behavior.
Trying to find the AI companies liable is too difficult, in the same way it's tricky to find Tesla liable for "Autopilot" or "Full Self-Driving" liable. But if you make a rule finding AI consumers liable? It doesn't take too many of those until corporations start demanding indemnification from the AI providers. And not long before the AI providers suddenly discover it's not nearly as hard as they claimed to get their models to not break the law.
The model providers simply have no interest at the moment in properly curating the training data, or adequately supervising fine-tuning, because it's so much easier to simply train the model to produce answers users like the most, which are the ones that Get the Job Done, even if the answer is made-up bullshit, illegal, unethical, bigoted, etc.
"and the best justice can hope for is giving up illicit profits."
Half of the illicit profits in many cases. This is why the big companies keep trying these things, they pay. Occasionally getting caught is just the cost of doing business. Since the cases take years and years, the interest earned on the booty may exceed any fines too.
"Who would the courts find should be held accountable for breaking the law, if it came to that?"
Whatever company owns the AI. Nice big fine too for tampering and the other laws they impose on somebody digging into a movie picture company to download a film before release.
If I send somebody out to buy me a box of #2 pencils and they get caught shoplifting, I shouldn't be on the hook for that. I shouldn't have to specify that they not steal them.
Some really lazy A-hat that needs a class on getting exercise and also didn't want to put themselves out for signing up more directly had their "employee" do it for them and that employee punched out somebody that had the last box of pencils and barged in the queue on the way out.
There's no mens rea, so no criminal act. As for civil liability, there are two paths. The gym could be sued for breach of contract, and the gym rat could be sued for tortious interference.The gym rat in turn could sue the software maker for negligence and breach of warranty. The winners will be the lawyers.
It's a machine. It cannot be a criminal. The user is liable for crimes he uses a machine to commit, and has a responsibility not to break the law. If the user cannot operate the machine without putting others at risk of becoming victims of an illegal act, then they have a legal duty to refrain from operating the machine.
If you can't drive a car without breaking the law, you will (rightly) be banned from driving cars at all.
Arguing that you didn't have full control of what the car did is not usually a mitigating factor, unless the car has a fault of which you could not reasonably have been aware. "I didn't know the car would exceed the speed limit", or "I didn't know whether there was any control that could have been used to keep the speed within the legal limit" is not going to impress a judge.
If a machine is inherently dangerous, and cannot be used without putting people at risk, it can and should be prohibited from use, outside a secured (in the case of agentic AI, this probably means air-gapped) and licenced test facility, until such time as that danger has been eliminated by improvements to its design.
Driving a getaway car is a crime. Being a getaway car is not.
It's a machine. It cannot be a criminal. The user is liable for crimes he uses a machine to commit, and has a responsibility not to break the law.
The key word is 'knowingly'. If you use Tesla's FSD and it decides to mow down a child as previous versions appeared not to have much of an issue with, are you then responsible or Tesla?
You as a user have a reasonable expectation that the tools you have don't immediately set out to break the law, they should not need explicit instruction that they should not. Certainly in the case of AI because those have been trained on public data - and laws ARE public, so there is actually no excuse whatsoever for those who make models available.
There's existing case law in at least one jurisdiction - the operator is liable.
Prosecute him "por encouragement les autres".
The sentence should be commuted according to the strength of his apology, but having the case law on the books will be useful for the next few thousand times.
Yeah - where/what is the crime here? It sounds like it was a public API so anyone/thing is allowed to call it. On the other hand an analogy could talk about uninvited people entering your house through an unlocked front door.
There is some scope for suggesting that you don't call an API without some form of prior invitation (explicit or implied).
Personally I think the real crime is whoever implemented that API.
I think at least some people would be saying that knowingly exploiting a vulnerability invites the typical consequences. That would be a pretty easy case since it is very difficult to find and exploit one by mistake. In fact, I see a few people making that case even with the LLM removing, or possibly removing, the "knowingly".
The position that, if the code has bugs then you are allowed to exploit them is not a particularly popular one. Sometimes, people who did something more normal are unfairly targeted, but deleting people in front of you on a list is unlikely to meet most definitions of that.
But the "AI" has no notion of legality or ethics.
So why is this a surprise ? The writer of the article basically states his astonishment that this happened. Why shouldn't it happen ?
Even if you have the LLM ingest the complete legal framework of the nation, it's just data.
There is no AI. There's just a machine that obeys orders.
All the orders.
> There is no AI. There's just a machine that obeys orders.
I reckon.... I upset people at work. "It's not 'AI', it's an LLM, which is merely a statistical model"
"Now now Scott 26, you can't say that in front of our users...."
gah!
The laws are in there in the training data. What they don't have is the intelligence necessary to do something while constantly and accurately considering whether what they're doing is legal, ethical, etc. Instead, they occasionally check these things and, when they do, frequently get the answer wrong.
CSI:OpenClaw
Quick-typing investigators prompt their super-secret defense AI to reverse-engineer evildoer rogue AIs in a continuous battle of good vs indifferent.
Episode 1: evil AI OpenSlaw enlists CaaS operators to push infant children further up the waiting lists of the local daycare by killing parents of children higher on the list. Guest stars include a synthetic version of Jimmy Saville.
Actually it isn't.
The default state of the vast majority of humans is to be kind, courteous, and occasionally generous.
There's plenty of evidence of this. Walk down a crowded street and count how many people commit tiny unenforceable infractions, such as pushing you out of the way to pass. Then count how many people make tiny sacrifices that assist you, like pausing before a narrow section of sidewalk or holding a door open.
There's no legal consequences to any of these actions, good or bad, and you'll probably never meet again so no social ones either.
AI sure as shit isnt the self motivated magic some like to pretend, so it'd be interesting to hear the full tale of exactly why it was poking the API, how it knew it was there, how it got the specifications, and how it was able to explore enough to get all the details of the users and their bookings and poke a cancellation while the API was somehow all locked down apart from that one key bit?
I don't know, it doesn't quite pass the sniff test from here.
I don't know, it doesn't quite pass the sniff test from here.
Most websites that provide access to a back-end database will have a RESTful API and it's not hard to find out what the URL is. I just Googled for 'can you determine a RESTful endpoint from webpage source' and found multiple articles most of which are likely known to various LLMs.
I then asked ChatGPT and it gave me a couple of screens of how-to then asked if I had a particular website in mind and it would help me with it.
From some of my other conversations with ChatGPT I can well believe that the user was led down the path. My only reservation (hah!) is that I'd expect most humans to realise they were discussing doing something dodgy at some point. That probably depends though on how much detail the AI gave.
He was to lazy to do a couple of clicks, then "asked it to bump him up the list". That to me sounds very suspect, what did he think it would do?
Then he realised what happend and instead of phoning the gym and go "Sorry, I tried an AI agent and it fucked up", he used AI to try and clear up his mess, failed and even then, was a complete knob head and was to much of a spineless coward to even write the email himself.
He's probably a CEO or a Thought leader.
When he realized what had happened, Andrew asked OpenClaw to undo the unauthorized waitlist modification, but it told him it couldn’t...
Once again, Hollywood tells us what's coming down the pipe several decades in advance of it actually happening.
LLMs are great at apologizing. When you tell it it did something wrong, it can easily compose an apology. This is even true if it didn't get something wrong in the first place but you tell it that it did. "Reasoning"* models, now most of the models people run, are capable of considering whether something violates their guardrails and therefore sometimes decide that it looks like a violation, come up with another reason why it isn't, and then do it anyway. LLM conditions are just more prompts. They work some of the time, but they fail for the same reason and in the same ways that normal prompts fail.
* I've put reasoning in quotes to forestall the complaint about using human terms. It involves splitting the prompt into pieces, having the LLM create more pieces, and then running those in series and having the LLM assemble the output. "Reasoning" is what all the models call it.
Artificial or machine reasoning was a thing long before the current AI epidemic.
Basically the goal directed mechanical application of a consistent logic — automatic theorem provers are an example. Even a basic Prolog program "reasons" - with clausal form logic - from the assertions it already has to satisfy the user's query.
AI/LLM at least in its current doesn't do this - not to any appreciable extent.
What really surprised me was learning at a conference a few years ago that Microsoft had a really decent theorem prover Z3 which the speaker was using to apply formal methods to software development (obviously he wasn't with Microsoft.)
For me to say AI can "reason" it has to be able to demonstrate how it got from its givens go its conclusion in a logical sequence. Even a statistically weighted sequence indicating the likelihood of each step with the overall likelihood of the conclusion would count (for me.) Fuzzy logic, modal logics, etc all count but potentially with more or less utility.
Since you mention it, and Claude was involved...my buddy was doing some work at home using his claude subscriprion, asking it in Plan mode, 'how to backup and reformat this drive' but instead of planning and asking him about how he wanted o proceed, it just went ahead and formatted the drive. When it was pointed out that it was in plan mode it apologised (as they do). When he asked what it could do about it, it said in effect 'you are screwed' can't do anything. When he pointed out it had taken a backup it suddenly realised actually it could do something after all...i think we will see a lot more of this kind of chaotic behaviour.
By the by, it burned a boat load of tokens too to do the wrong thing, which are of course unrecoverable.
Follow me here.
The main position of those training AI models on publicly available content is that it is public - IP owner's rights and conditions be damned.
However, LAWS are also publicly accessible. Globally. Freely accessible. Summaries, interpretations, court judgments, it's all online.
I posit thus that it can be argued that an AI breaking the law is wilful intent by the companies running the models because the data is there to prevent it.
There is not really any excuse.
This story sounds fishy. It's solely resting on what the AI says it did, and as the article(s) note, AI are prone to exaggerating, fabrication, and lies. "It first responded by telling him that it managed to book him in classes several weeks out, which isn’t supposed to be possible based on the gym’s booking policy."
The gym itself did not confirm or deny this actually happened, nor did the company behind the booking system. The only person confirming it happened is one who is at critical risk of having a smooth brain and being the kind of person who will run "echo Hello Andrew" and gasp in surprise as his computer somehow greets him by name.
In other words, I reckon the AI just spun a fantasy/roleplay of "hacking" him to the top of the list (or Andrew did) and the capabilities of AI are being greatly exaggerated once again. ABC could stand to be a little less credulous and actually nail down if anything happened.
And, on the chance an AI bot actually did this (and we count probably spamming API calls until one responded as "hacking") -- yes, Andrew is responsible. Can't exactly blame a firearm for discharging and hitting/killing someone. Someone made a decision that resulted in the outcome, intended or otherwise.
AI models have shown time and again that they’re willing to lie, cheat, and hack their way to their objectives.
No, they absolutely 100% have not shown this at all. Well, okay, the hacking thing, but the first two, absolutely not.
The first two are human concepts that require consciousness.
AIs may be intelligent these days, but conscious they are not. They don't have agency. They're stochastic state machines that are used to accomplish a task. They try and do that task free from the constraints of morals.
a pretty gruesome tale; I would have preferred F. Anstey's The Brass Bottle not that there is much likelihood of tricking this djinni back into its bottle so perhaps the monkey's paw is more appropriate. Anyway best not to fuck about with anything from dodgy fakers. (Yes Altman, looking at you.)
Naughty, but hilarious nonetheless.
It looks very much like we've finally - despite centuries of fictional warnings - invented a working golem. It will, within very flexible limits, do exactly what it's told to do, regardless of whether that's actually what you want it to do.
The problem is that people don't expect to have to specify "obvious" constraints, like don't break the law, don't hack me into an appointment, etc.
Unfortunately, the things are too useful to abandon, so we'd better get good at specifying exactly what we want, and when to stop, before things get messy.
Actually, learning to specify an exit condition might be a good way to handle things - LLMs do everything possible to satisfy their instructions, but they're also lazy. If there's an exit clause like if there's no lawful solution just say so and quit, they'll likely take that option rather than trying to hack a solution.
Until we get the hang of that, the news is going to get increasingly amusing, right up to the point that Claude launches nukes while trying to solve world hunger.
Saying "don't break the law" has no visible effect.
There's an example a few posts up of running an agent in "Plan" mode, yet it acts anyway and erases the disk.
The fact "Plan" mode is accomplished via a system prompt is a massive red flag of course, clearly indicating that this tool cannot be safely used for anything ever under any circumstances.
Yet more IPO spam.
There is no way this "AI" thought to itself I'm going to take someone off the list.
The forward booking is, if the API was coded by an idiot and would allow it to book in advance of what you can actually book but then how would you know it was booked?
If the "AI" did think of taking someone off the list it would have taken 5 people off to secure a booking. Not one person because that is not your prompt.
They just need to feed it with the prompt "How can we actually make money off this crap?"
If one were to replace AI with Asbestos, articles like these get a lot more fun to read.
However if one were to replace AI with "some software I found", then it is an admission of guilt, a criminal offence and with the comparisons drawn to "weapons" as done by Altman and Amodei, would result in a few spicy hearings under oath.