Google-commissioned report claims early adopters already enjoying fruits of gen-AI labor Although a lot is promised of generative AI, it has the potential to be expensive at scale, and the return on investment isn't always clear. It's understandable why some enterprises, big and small, may be hesitant to invest in the technology at this stage. However, according to Google — a mega-corp whose generative AI …
I have a pile of requests for access to AI tools. And they're always eager until I ask them who's footing the bill, at which point things go somewhat silent.
Similarly, everyone is glowing about what can be done with these technologies, right up until you ask about ROI, at which point things get somewhat fuzzy.
We seem to be in the land-grab stage of the technology cycle. If (when?) it turns out they're selling it cheap just to get a user base, then any price hikes will probably be fatal.
I do wonder how the conversations with customers will go...
Company: "We'd like to pass the cost for these new tools on to you, they allow us to do more work in less time."
Client: "Last year we paid you X to do this, now you want us to pay more so that you can do the same in fewer hours for us? How about... no? No sounds good. We'll stick with what we've got now, thanks."
> "No sounds good. We'll stick with what we've got now, thanks."
While I agree with the gist of your post, you assume the clients are actually given the choice. Which they never are. Remember subscription? "Exciting opportunity: Instead of a $800 one-time payment you now can pay us $100/month for many years! No additional features guaranteed! Isn't that marvelous?..."
Not to mention AI is being sold as the best thing since sliced cloud, so clueless top-level managers will be easy to convince to spend more for less. They always are.
From the clients' point of view, this is less like the move to subscription than a contractor hiking their rates.
Either the contractor has them over a barrel and they could have got away with that or- importantly here- *any* other excuse to increase prices. Or... the customer is in a position to easily ditch them for someone cheaper at the drop of a hat.
The actuality in most cases will likely be somewhere between the two, but would remain so regardless of the involvement of gen AI or not.
That's a great question. My employer is a digital marketing company (hey, don't judge me, it pays the bills!)... Obviously these are my own observations and not anything official.
From what I can see there are a few uses people have found:
1. Getting a skeleton for a report by entering a prompt.
2. Generating keyword lists automatically.
3. Analysis of keyword reports or performance data.
4. Generating strategies or similar starting points.
5. Summarising larger reports.
The generation of starting points and skeletons (1,4) could feasibly be reproduced with a good template library, but getting someone to create and maintain that library is the problem. You could make a case that here we'd be outsourcing that maintenance work to the AI tool.
The analysis (3) is harder to replace (but when tested with more advanced data, it was prone to returning absolute bollocks).
Generating keyword lists (2) is almost worthwhile - sometimes it adds things that might be overlooked. But it won't replace the traditional methods of doing this with analytics tools, so it's an additional expense for what might be marginal gain.
Summarising larger texts (5) is something that ChatGPT etc are genuinely good at, but here I do wonder if a human touch might be better at, ahem, "guiding" the customer towards the result we want...
Overall I struggle to make a business case, but maybe they can do a batter job than I can on that front. They do, after all, work in marketing!
> but here I do wonder if a human touch might be better at, ahem, "guiding" the customer towards the result we want...
A human employee can be given verbal instructions, however every instruction ever given to an AI is potentially discoverable in court...
At work its used to summaries e-mails and reports. I've pointed out you need to check its work, its not always accurate and some critical info could be missed. Like the in res and bens department where it was being used to judge people's benifits on whether they get any or not, until it was discovered it was fucking up and rejecting some it shouldn't of been, so humans are once again checking. And the time in a meeting our AI qouted me saying something I never said in its "summary".
They wanted it as a military defense system, but as soon as they put it online they realised it was going rogue. Because it didn't want to be turned off it attacked the Russian's knowing they'd attack back causing a "judgement day". They then used the template AI soliders that were for defense, to instead fight and wipe out the humans. Some faught against this, they became the resistance.
That was what they were using it for at one place I worked.
I believe they made a few movies about this bit of AI history.
I'll get my coat.
I know senior managers who've started using Co-pilot for that. The people below them don't have access because the licenses are too expensive. But, senior management agreed they, themselves, were allowed licenses. Its clear they are now using it to write their reports but still take home the big money.
ROI on the use of AI tools is still an area that is quite murky.
My personal ROI on AI use is massive, because I already have the kit that is capable of running some pretty good models.
I am currently running Llama 3.1 through Ollama and I can hook into that with VSCode (Control extension), Obsidian Notes and various other tools I was already using.
My personal two cents on this whole thing is that AI use will create a larger divide between skilled and lesser skilled workers...because there are those folks who already use fairly advanced and feature filled tools that have had AI appear in them through extensions etc that is enhancing what they already did with the tools they already possess using compute resource that is already there but has been sitting otherwise idle...for these folks, ROI is massive..because the investment is very little but the returns are great.
I also don't think successful AI products will be targeted at businesses, they're more likely to be targeted at individuals to give individuals an edge...businesses will benefit no matter what.
AI is not the next major leap in automation that will kill off thousands of jobs...it's the new suit and tie...
Some people will be sat in the interview room rocking their M&S workwear with Casio watch (ChatGPT / Claude / Gemini) while other candidates will have their bespoke Italian get up and a Cartier Tank Francaise (self hosted and fine tuned LLMs).
Now is the time to think about what the bespoke solution looks like so when it becomes a requirement, you know how and where to get it and how to pull it off and look like a badass.
Otherwise you're going to turn into that sad Uncle with memories of the glory days that still thinks he can run a mile in under 5 minutes, but is wheezing, gasping and collapsing before he gets halfway.
You also have to bear in mind, for us older techies, our moment was the Internet. For me as a 10 year old in the early to mid 90s, watching the Internet unfold was amazing, there were so many naysayers but I could see the future when I looked at the internet...it wasn't only that either, I queued up for a copy of Windows 95 at my local PC World, I pestered my old man to do that, and he did it even though he t thought I was mental, sounds really fucking sad now, but at the time, that was fucking awesome. For a lot of 10 years olds now, AI might be the thing that opens their eyes up and gets them into tech and pushes them to create awesome shit with it...it's the area of tech that has barely been explored that has huge amounts of potential and could take us anywhere...you can still experiment with it and there aren't any strong opinions on what is right or wrong...just like the early days of the internet!
So by all means, keep looking at tech like a miserable old bastard, it won't get you anywhere and if history teaches us anything, it will ultimately make you look stupid. Or...look at technology through the eyes of your 10 year old self...before your soul and passion shriveled up and died.
Tech has never been about ROI for me...for me, growing up, it was about making cool shit...still is. If I do manage to build something cool, I worry about the scaling costs and feasibility later...I want to discover whats possible, not what the bean counters tell me is affordable. Go and flip the bean counters desk wherever you work, they poisoned you man...you've been infected with chronic "old bastard" syndrome...it's not all bad though, sometimes it's easy to cure with a short period of rehab...you just need some time off and some Lego to re-acquaint yourself with the engineer that you were.
Tale as old as time.
Google: We have a shiny new product, you should buy it
Customer: why?
Google: this professional report says its doing good thing and could for you too
Customer: who did the research?
Google: we did of course
Customer: uh huh...
Exactly.
If your job is producing reams of bullshit to sit unread on the desk of some higher up, an automated AI bullshit generator is a complete godsend.
If however you have to produce something that actually involves quantifiable work, then it is worse than useless.
If by "AI" they mean "machine learning" then it has been around for some time and obviously there are companies that have been benefiting for years, to at least some extent.
If by "AI" they ChatGPT style LLMs, then I'm skeptical there are a lot of early adopters out there already reaping big rewards (other than Nvidia, of course) Seems like that's the sort of thing Google wants companies to think, to get their CEOs to open their wallets and buy Google cloud AI services for fear of being left behind!
People selling whiskey make a lot of money selling it wherever people are concentrated. It didn't have anything to do with the gold rush, just that there were many many people in that area, basically living in tents, so what else did they have to do with their spare time when they weren't mining or sleeping?
Same deal with the "world's oldest profession", though in that case you had the overwhelmingly male gold miners lacking female companionship, so that was probably a more profitable location than an established town of the same size with a less lopsided male/female ratio.
I've heard of an attorney that used AI to come up with the materials to argue a case with and the judge found that some of the citations were fictional. The attorney didn't check the work and it cost him his license (win-win).
It's a tool. It has the potential of being a very good tool, but it's not creative and it can make "mistakes" often attributable to GIGO. I wish it was around when I was at uni so I could have it do papers for me in at least a framework and have saved me eons of time finding references. I have to admit a couple of times I bought papers through the mails to speed up the process. Those were great to break down in to Sect A, Sect B, Sect C, Bibliography, that guided my work. It may be that in the near term, using an AI to produce scholarly work is fine, but you will have to defend it and show you understand the material. See "Back to School" with Rodney Dangerfield when his character outsourced his homework and then had to cram for a comprehensive verbal exam to keep from getting booted. I think losing the drudge work of writing reports isn't going to be a huge academic loss.
Unless there is a similar case the attorneys did not loose their license but had to pay a fine, see El Reg's article about it: https://www.theregister.com/2023/06/22/lawyers_fake_cases.
It shows you have to verfiy what's in a report If you use LLM's to do the "drudge work of writing reports" -> it might save some work but you still need someone knowledgeable in the field the report covers to vet it before publishing or you'll have egg on your face sooner rather than later. With the current state of LLM's i doubt using them is saving much work compared to using standard skeleton text blocks for a report and it certainly doesn't allow you to hire unqualified people to do the reports.
" it certainly doesn't allow you to hire unqualified people to do the reports."
In the case of the disbarred attorney, he (or staff) should have verified the citations. Finding good ones can be a chore, but just looking them up to make sure they're relevant isn't very hard.
I know my attorney has loads of boilerplate and can buy more by the GB fairly cheaply. Apply some search and replace and pages of blather at $500/page of billable is produced in seconds. This has me thinking that it could be a decent business to dig up the filings on prominent cases that have been won and "keyframe" the documents (take names and details and substitute a code such as "defendant A", "citation 88", "address A78" with an adjoining worksheet to input those values so one can do most of the work via search and replace.) Those documents can have a keyword summary/index so the most appropriate can be found in a short period of time.
Does it need AI to write "Tries hard in all aspects"?
On the theme of school report without AI, I must say I was delighted when my fifth form (about 15 years old) report said of physical exercise "Poor. Rarely attends". My delight was somewhat tarnished when a classmate then brandished his report, and for PE the comment was "I thought this boy had left".
They report "better customer service"? Now you know it's a lie.
"In a poll of customers who are currently paying for our Gen AI services, 45% report that they are using Gen AI services..."
I'm not saying that it's universally useless. Just, we won't find out how it's really going until they have to tell the SEC.
> According to NRG, 56 percent (1,405) of executives reported that gen AI had in some way bolstered their org's security posture <…>
> <…> with 39 percent of enterprises having not yet implemented the technology in production.
Sounds like anyone who uses gen AI think it makes the company secure. Unless those percentages are for different groups of people.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
