6,000 execs struggle to find the AI productivity boom A survey of almost 6,000 corporate execs across the US, UK, Germany, and Australia found that more than 80 percent detect no discernible impact from AI on either employment or productivity. The study from the National Bureau of Economic Research (NBER) in Massachusetts is based on input from CFOs, CEOs, and executives at …
Wondering who TF they surveyed ??
“and 89 percent saw no change in productivity (measured as volume of sales per employee).”
Well that ignores huge sectors that doesn’t sell anything - Government, much of Banking/finance, Healthcare (US aside), Military, Utilities.
“The report also notes that employee expectations differed from the senior execs, in that workers expected to see more jobs created as a result of AI over the next three years, along with smaller productivity gains.”
Complete bullshit. The thousands tossed from Amazon.’, Salesforce would disagreee with that.
If A.I. improvements mean you need 4 days to do 5 days work, you will get an extra days leave - unpaid. This will squeeze jobs if it ever materialises.
I don't mean to be rude, but what planet are you living on? Tens of thousands of people have been let go because their corporate overlords preferred to micturate away their cash on hand on AI "data" center builds to spending it on, you know, human employees and their benefits packages. An economic shift so far proven good only for a very small number of GPU chip makers, memory purveyors, and the fossil fuel extraction industry.
And even if that hope materializes, that would mean complete doom for Investment-grade AI as we know it, namely as huge models hosted remotely in gigantic data centers built with insane amounts of hardware and operated with insane amounts of electricity.
Prices extracted from customers for a 1,4% productivity rise in 3 years will never pay for the electricity consumed by modern AI alone, let alone for interest on all the investment.
That would simply mean Game Over for the old AI guard and would shift industry focus to smaller, more optimized local models capable of running on edge systems, that have a chance to become economically viable.
Beneath the weird appropriation of Japanese myth and legend, the basic promise of increasing productivity by culling the least value staff had a basis in something.
That *something* was effectively a pyramid scheme, dependent upon being able to find higher value staff.
But, pyramid scheme's work for the early adopters.
So, 90% see no ROI in their AI. That means 10% do.
Assuming AI is just a standard hoax; of that 10%, maybe 20% (or 2% overall) actually benefit.
Lets say they benefit *alot*, like 2-fold improvement in business.
Also, that the other 8% see a 50% contraction of their business.
So, 8% becomes 4, and 2% also becomes 4.
Thus 10% of the market shrinks to 8%, leaving the 90% of AI-skeptics to gain 2% of business, merely by attrition.
Ergo, by 10% of the market going balls deep in AI; 90% of the market sees a 2% improvement; 8% of the market surrenders 50% of their business, and 2% double their business.
Oh, and it didn't cost the 90% a penny; whereas the 2%, and obviously the 8%, may have questionable ROI.
Or more likely, the 10 percent of businesses that say they had an increase in productivity don't have a good way of measuring productivity due to AI and if they could measure would find that they weren't seeing any more improvement than the other 90 percent.
The first thing you have to do before you can measure an increase in productivity is to be able to measure current productivity. Most businesses can't do that with any degree of accuracy when it comes to white collar office jobs. A lot of companies seem to be relying on surveys of employees to ask them if they "feel" more productive. That is such an unscientific measure that it's laughable.
A 1.4% increase in productivity for office jobs is quite frankly statistical noise.
If there were any genuine success stories the management consultancies would be publishing loads of case studies showing double digit productivity increases year after year as AI reaches ever further into businesses. We're not seeing them. The only scientifically based studies that I have seen are showing no or even negative productivity effects.
The reason that LLM AI is being called a bubble is not because the technology doesn't work at all. It's being called a bubble because the useful results do not justify the amount of money being sunk into it. The mountain has groaned and brought forth a mouse.
I'm thinking back to my industrial engineering (and before that work study) days. Measuring productivity on a machine meant to produce 100 widgets an hour is relatively easy. You can pretty much ignore the cause of the productivity gain/loss when measuring.
Moving away from direct production its far more difficult, especially when you hit those jobs which support neither sales nor production.
That almost sounds like an increase in productivity UNTIL you realise your costs have increased and that nice jar of jam which you picked up from the shop on the way home costing £1 is now £1.75 to pay the deliver costs and the director or the deliver firm's bonus
What one client has noticed is the increased use of AI to review bids.
They witnessed a significant downturn in the number of bids they were winning. A colleague spent sometime investigating and discovered the use of AI in the proposal evaluation. He then used AI to rework some proposals, used the knowledge gained ie. The phrasing and language used by the AI and applied it to their proposals and increased the number of successful proposals…
So it looks like whilst AI is actually adding nothing of any substance, without it you won’t remain I business.
AI is also being (mis)used by Human Resources departments to triage applications for jobs. The problem here is that a job advert will often garner hundreds of applications, of which a dozen or so are worth interviewing and maybe a handful are actually suitable for the role. The trick is to cut down these numbers by some form of filtering, and AI promised to be the tool for the job.
It isn't. I don't know what is, but AI isn't the tool for the job.
Comic Sans is demonstrably more readable than Times New Roman making it a good choice if you care about the greatest number of people being able to read it.
So, yes, Comic Sans - and newer fonts that have taken its characteristics to heart - are much better than TNR.
Of course, there is all the social baggage attached, to both fonts, which leads to so much sneering. But social whims *can* be changed, so when you hear somebody deriding anything that reminds them of Comic Sans, just say the magic words: for f@#£'s sake, just grow up.
I don't know what is...
Meat bags who understand the job, which is hired for. But it depends...
It's quite simple. Anything you consider key to your business, anything which distinguishes, differentiates your business from others, anything that gives your business a competitive edge: you do yourself. And "do yourself" means doing it inhouse with competent employees.
So, if you consider your staff a commodity, do not hesitate to delegate the candidate selection - heck, the whole hiring! - to some AI. On the other hand, if your staff is crucial for the business' prosperity, invest in a hiring process that selects the best matches for the job and do not try to cut corners with a mediocre procedure that is likely to dismiss good candidates.
I can't be bothered to dig out the studies now but I seem to remember that some research showed that from the set of suitable candidates*, a random choice to hire wasn't less successful than all the interview and assessment BS. That way you don't reject the unlucky ones, you choose The Lucky One.
* that is, after weeding out the rotten ones like fake applications or the MBA applying for an engineers' job.
From my time in recruitment I discovered that most people were honest and if you paid them the compliment of being honest with them the unsuitable ones would remove themselves from the process. However, you always got some that felt they were admirably qualified to be PM when they should have been the bog cleaner.
The problem is more fundamental: it is using HR people do make any decisions at all about whom to employ rather than people who actually know what the work involves. The very most HR should be given to do is issuing contracts and other paperwork, and never anything which requires decision making.
In theory, GDPR allows the individual to ask for the data and algorithm behind any automated decision made about them. Given that any A so-called I system cannot do this, I would love to see someone take this to court. Admittedly, this might lead to the company not hiring someone to say that they had information that led them to believe that they were the sort of tetchy wanker who would take them to court.
Louis Rossmann made a video about his website (i.e. livelihood) dropping in Google search rank and having to use Gemini to rewrite his website into bland nonsense as nothing else would work. It got back up to number 1.
I guess for whatever reason the number of AI slop websites are affecting Google's search rank because the majority of new websites on the Internet are now AI slop so to be successful your website also has to be AI slop.
1.4 percent is statistical noise.
There was a famous experiment done during WWII in which an industrial engineer was trying to measure the effects of changing factory lighting on worker productivity. However, he could not find an optimal lighting level. There seemed to be no pattern to how the inputs affected the outputs.
Eventually he figured out that any change produced a temporary increase in productivity for psychological reasons. However, that increase faded away and returned to its original level after a period of time as people became used to the new conditions. Within certain reasonable limits, the actual lighting level had no effect.
This is an example of how productivity measurements need to have a scientific basis or they're worthless. The exception is when the productivity increases are so large and unmistakable that there is no question there has been an improvement. We're not seeing that with the published AI data. It'd down in the statistical noise levels.
I have a long background in factory automation and every productivity project that I've worked on had a clear and measurable goal measured in terms of actual money, and success was based on meeting targets. If productivity increased but by enough to meet the minimum return on investment criteria, then the project was considered a money losing failure.
I haven't been seeing this rigorous measurement with AI. People are throwing AI at their offices with no clear idea of what they hope to achieve or any rigorous method of measuring it. It's farcical.
There is one possible gain. An AI avatar to attend meetings. As standard it would be prompted "If they want anything from us find out what it is and say we might get back to them. Otherwise make occasional encouraging but non-committal noises but don't promise them anything". Then the meeting can go ahead without interrupting work. Eventually everyone uses an avatar, meetings are no less effective than they were before. The only ones attending were those calling the meetings because they have nothing to get on with instead; they can be culled.
I've been looking into this in some detail.
The chances are that most people are doing AI wrong, in the same way that we've always adopted new things inappropriately.
There are 4 approaches that organisations could take:
1, Throw AI licences at employees
2, Buy the latest versions of their existing products with AI embedded in them
3, Redesign their internal processes so that they are oriented towards AI usage to improve it end-to-end
4, Rethink their customer facing experience and either add to it (chatbots, conversational interfaces etc) or empower the existing experience with AI (improved personalisation, better product selection, etc)
I suspect that almost everyone is doing 1 & 2. It's simple, quick, and relatively painless. it's also distracting as people faff about with this new widget without proper instruction on how to use it.
History shows that neither of these produce any measurable results at a business level as it improves a task but not the overall work throughput. For analogies look to the early stages of electrification of factories, early stages of the wide spread use of computers, internet adoption, etc.
C&D is where ROI can be shown, however this is more disruptive and people are less willing to throw cash at big change without big proof of success, which is not yet visible.
With respect to 4 I can see this being taken up enthusiastically be companies whose customer-facing operations are already dire and are seeking to not improve them at lower cost.
They're the sort of people who would list things from 1 o 4 & then choose C & D.
Thanks for spotting that. I had it in my memory from a different document laid out as A-D but rewrote it here with 1-4 for a less commercial audience.
We ridicule AI today for all it's faults. But this is no more sensible than ridiculing the intern for not knowing how you work, whilst ignoring that they have just arrived with a completely different approach.
I get very frustrated with using AI, but I know that I am not using it correctly from advice I've received from friends. Don't ask it to carry out large tasks. Ask it to write scripts or software to carry out large or sequential tasks. Ask it to verify that it has produced against the original request. And so on.
I caught up with two friends in the last week, one who runs the AI division of a system integrator and another that runs IT programmes for a global enterprise. Both had live examples of enterprise applications based on SaaS being ripped out and replaced with bespoke written software using AI coding tools. Not PoCs, but live, mission critical SaaS applications being decommissioned and replaced.
In one case this took 6 weeks, in the other 16 weeks. In both cases, it was a team of 5/6 that wrote the software.
I'm not fanboy for AI. The impact that this will have is going to be disturbing at the very least and already is. But now the genie is out of the bottle, it's a case of whether you want to be squashed by it or try to eek out your career a bit longer by going with the flow and trying to do our best with it.
You nicely assume that options 3 and 4 would work without any attempt to prove them. Are we supposed to believe that nobody at all has tried that?
To redesign their processes, they have to know where AI would help, which would mean they'd need to know what they can count on AI to do correctly, which is a big problem with most approaches. I've seen experiments trying to do that. Hallucinations or models that can't get or process data have caused problems for them all.
Stuffing AI into the customer experience is similar. I've seen plenty of chatbot customer service. It's different from the previous automated chat-based customer service. Those chatbots just never worked; you'd type something and it would either not respond or give a single canned response. The new version pretends to answer your question, but not usefully. The rest of your suggestions are so basic that someone should be able to build that about two years ago. Where's the LLM-suggested product system, which should be easily built? I don't see that. Maybe the reason I don't see it is that, when I've seen it tried, it couldn't produce an accurate comparison between some similar products I didn't want to have to make a spreadsheet to compare because it didn't have some important information (I had to read the manuals to get it and evidently nobody thought to put those in the bot) and, when it did, it frequently made me fight to get it because it kept trying to simplify technical details which, when buying something, I want to know specifically. Maybe companies tried that and realized it would put off customers instead.
I'm not assuming anything really.
Just looking back at history of computerisation in the 70s, 80s & 90s, electrification of factories in the 1800s, The rising use of the Internet post 2000. They all show the same approach.
Stage 1, just give everyone the new thing.
Stage 2, update their existing tools to include the new thing
Stage 3, Re-think how your business works internally so you can take advantage of the thing across a whole business process or processes
Stage 4, Re-think how you interact with clients using this new thing
It's not necessarily sequential, and almost certainly shouldn't be. But the stages are pretty easily identifiable and the reasons that 1 & 2 don't have the desired effect are consistent.
To your point about whether an LLM is the right answer to a given problem is completely fair, and yet more evidence of using AI inappropriately. A little like asking an LLM how many Rs there are in a given word. The useful approach is to ask an LLM to write a script that can determine how many Rs there are in any number of words which it will do very, very well.
In exactly the same way that we don't ask programmers to count how many products are passing through a factory., We ask them to write software that measures this repeatedly.
In two prompts I was presented with a LinkedIn post formatting tool. Literally 3 minutes work. I will never, ever pay for a tool to do that now. But I would not ask an LLM to format my LinkedIn posts as I know it would do so in a non-deterministic manner.
So, LLMs are good at some stuff, but not at all.
The thing I think most people are not really paying attention to is that AI does not need to be perfect to replace humans in a given scenario. It just has to be good enough at a given price point for it to be a viable alternative, given it will work 24x7.
I think we're going to hear the term 'good enough' quite a lot in future.
The problem for the big players in the LLM market is they need to increase their prices at least 5-fold and probably more in order to start getting a return on their investments (including paying off the horrendous debts they are accumulating). At the moment the more active customers they add the higher their losses and the higher the debt. So they desparately need to lock in as many large companies as possible as quickly as they can. Until they reach this point they need to be able to continue convincing investors/lenders that they're backing a future cartel member in a hugely profitable market.
The problem for companies building products on top of LLMs (either for internal use or as products) is they are vulnerable to price increases and to updates that change how their products behave; for commercial products they also face the risk that the LLM supplier will create their own version of any that become successful.
You go wrong at step 1. It should be:
Determine who's work will or is likely to benefit from the new tool and give it to a selection of them.
You then miss out a step
Check and see if the marvelous new tool has made any bloody difference.
I accept that sometimes other factors influence decisions eg the FD's (or CFO's) secretary (or PA) gets a posh new computer rather than the person who could make use of it, but that's life.
There is a flawed assumption - option 3 requires that an AI infused process is better / more productive than the current non-AI one. I'm also dubious about option 4. Please compare the experience between "talking" to an AI and talking to a knowledgeable human who is trying to be helpful.
I went to Expedia the other day to rent a car. Between them and the actual rental agency, something went wrong and I couldn't get a confirmation from the car rental agency that they had my reservation. So I tried the Expedia chatbot. But it seems to have only two answers to every question: show me Expedia's copy of my reservation (which is where I started) or cancel my reservation. No way to get to talk to a human being. Worse, when it offers you a link to their copy of my reservation and a link to cancel, those are the only two things you can do; you can't even type into the box, because it's blocked.
Whoever did #4 at Expedia (rethought the customer experience) needs to be fired.
.. Though that is what many companies are hoping for - a reduction in head count as a way to drive down costs.
I have already experienced "AI" customer service - it was worse than the usual scenario of dealing with a human following a script & outsourced to a cheap labour country (and why the F... do they insist on giving such customer service bods anglicised names - just use their proper names - I'm more happy to chat with Sahana*, than I am to "Sarah"* (when "Sarah" has an obvious Indian accent))
* Other names are available.
Those of us who have been round the block several times have seen this sort of thing before...
Before AI, it was 'natural language querying', which is a precursor in some ways, but have never seen a properly functional one. That was another damp squib, where the nuances tripped it up, amongst other things.
I've had a play with AI in querying a hefty database, and it needed so much hand holding.... worse was that when it was eventually coerced / coaxed into making sense, the DB was literally brought to its knees by the SQL generated.
They used English words in the same way that Fortran used “IF” and “GO TO” just more so. FORTRAN was designed to make it easier to implement mathematical formula in code. COBOL was designed to make it easier to implement business rules written in plain English into code. There was never any intention to compile English or to have non professionals code.
The language designed for non programmers was BASIC and very successful it was.
Well Musk says he'll put up one million satellites to provide compute powered by solar. We'll see how that goes, but it could be like the dark fibre era -- every man and his dog laying cables coz the demand will be astronomical, but it wasn't. Opps.
Still if people really believe it's just time to find that killer app, then look forward to cheap compute in about a decades time. :-)
A major problem with AI is not reflective. You give a set of instructions and out pops something that may be useful or more likely slop. Many real world problems require reflection and asking questions before developing an answer. So there might be a couple of rounds of questions and answers before actually finishing the project.
AI should be good at giving sort of correct answers for easily defined issues that have what are effectively 'Yes/No' answers.
The fun thing with LLMs, is so many companies not only WILL end up being breached because of them by putting in data they shouldn't, leaking data or realising how fucking expensive it is. For example CoPilot, lets use that on company data, it will make things so much better. Yeah but what they don't tell you, for it to work properly on your company data, you need sharepoint setup and all files tagged properly. So you now have to pay for sharepoint and an admin to manage it for you.
69% of businesses currently use AI. 20% of businesses see discernible impact from AI. For a technology that has been around for just three years, those percentages are great. IME of working in the B2B technology space for 40 years, I haven't seen better adoption or ROI numbers for any previous Wannabe Revolutionary Enterprise Technology - e.g. ERP, Web, Mobility, Cloud, Metaverse, Blockchain - at a similar stage of its product lifecycle.
But yawn, how many times do we have to listen to BS.
In the 70s/80s it was CASE tools, it will revolutionise programming and devs will be more productive/maybe obsolete.
In the 90s it was 4GLs and OO and 'RAD', it will revolutionise programming and devs will be more productive/maybe obsolete.
In the 90s/00s it was DSDM and Agile,it will revolutionise programming and devs will be more productive/maybe obsolete.
In the 10s it was microservices,it will revolutionise programming and devs will be more productive/maybe obsolete.
Yep I started in the mid 80s...coding.
Its 2026 and I'm...coding
Just GTFO.
The rush to monetize AI is both misguided and premature. Unlike tradtional computer software, where I worked for over 40 years until recently, AI is transformational at a societal level. Microsoft, Amazon, Google, Meta and other tech giants are investing 100s of billions of dollars with no promise of an ROI or even a solid business case. When I developed a computer application to automate a manual process or make and existing process more efficient I had to present the bottom line ROI for the project. The Business Side of project management kept us in IT from going off the rails. In turn, we provided a working solution that benefited the businees.I see no such business model for AI, which cannot already be addressed through existing software solutions. This appears to be a case of Fear of Being Left Behind. AI will arrive, but not in the way their creators and proponents envision. I retired from computers and software over a year ago. I'll be watching from the sidelines. Good luck and thanks for all the fish. Cheers.
Business leader hate admitting that they made a mistake. While you might get one to say "AI isn't providing us with the productivity gains we expected - yet", they will never answer the question with "I made a big mistake investing in this stuff, I wish I hadn't".
While we don't see any measurable productivity gains from our very, very expensive AI system, we have to recoup that expense somewhere or our stock options will tank.
Therefore, I propose a 25% headcount reduction spread over every department.
This will pad our quarterly figures so guys, get your ducks in a row and prepare to exit stage left before the year end results come out.
Signed
The CEO
{excuse me, I need to close on that $50M yacht}
"Donald Trump's pick to lead the Federal Reserve, Kevin Warsh, has signaled he will lean on artificial intelligence as a key justification for cutting interest rates. In a December interview with fintech entrepreneur Sadi Khan, he described AI as ushering in 'the most productivity-enhancing wave of our lifetimes — past, present and future,' calling the technology potentially 'structurally disinflationary' in the same way the internet once was." -- as quoted in Financial World (https://www.financial-world.org/news/news/financial/30304/kevin-warsh-argues-aidriven-productivity-should-open-the-door-to-rate-cuts/)
I guess Warsh needs some excuse to push his boss's views on interest rates.
