KPMG wrote 100-page prompt to build agentic TaxBot The Australian arm of consultancy firm KPMG wrote a 100-page prompt to create an agentic system that prepares tax advice far faster than humans. Speaking at analyst outfit Forrester's APAC Technology & Innovation Summit on Tuesday, KPMG chief digital officer John Munnelly said the firm's "life changed" when ChatGPT debuted in …
It sounds like KPMG covering its arse. They mean its output contains errors so esoteric and presumably liability-inducing that you need to be a tax expert to spot and correct them before letting a client or the ATO see it.
Then each client can have an instance of their AI agent arguing with an instance of the tax office agent about what they have to pay, without any expensive tax lawyers being involved. There's bound to be a way to monetize that,
Suspect the LLM was fed documents that had not been redacted, so the output would not only contain client confidiental information, but actual client identifiable and incriminating information (eg. By processing coffeee beans via Switzerland - tax avoidance, we can add an imaginary “handling fee” and thus increase costs in the UK - tax evasion.)
Also after having understood the importance of parametric database queries for decades, with AI we went back to feeding instructions and user-controllable input into the same channel, opening up a whole new world of entirely predictable security holes.
The big four consultancies have been under intense scrutiny in Australia over the last few years since it became public that PwC was acting in a way that can only be described as corrupt with respect to its dealings with the federal government through the ATO and its private tax clients. All four were eventually embroiled in scandals of varying seriousness once the spotlight was shone upon them. That might have influenced KPMG's risk-aversion in this instance.
Given the personal data was found by ChatGPT, it could reasonably be assumed that this data had been scrapped by ChatGPT during its learning phase and thus now part of ChatGPTs LLM, accessible to everyone with a ChatGPT account, ie. The data has already left the building and is in the public domain in.
So blocking internal usage of AI does smack of bolting the stable door after the horse has bolted, as if there is other sensitive information on KPMGs systems, it will have also been scrapped…
Obviously, further on, the article does talk about creating their own LLM using data scattered across their internal systems, which also further exposed just how poorly they managed data [aside. From my experience the level of data mess KPMG discovered, is probably the norm for the majority of businesses.].
I suggest the collation of internal data also introduces another security risk, client confidentiality and thus the potential for reputational damage. I suspect KPMG didn’t want to disclose that those files retained on individual laptops contained client sensitive information (company names, names of individuals, bank accounts, details of specific tax avoidance schemes and arrangements etc.). Hence the AI would have also disclosed the need for a robust document sanitisation and curation procedure for all LLM learning materials.
1. Does this new, faster method produce complete and accurate results? No.
2. Is this 100-page LLM prompt effectively-maintainable software? Probably not.
3. Does this smack of corporate-image-spinmeistering over rationality and logic? Yes.
Knocking out the work in a day rather than two weeks is great but in a business where billable hours are king clients are going to want to see significant savings. Consultancies will try to swing the billing justification away from total hours worked to value generated but will their customers be convinced?
The amount that you can charge for advice is related to the effort needed to create it. If partners think that they can reduce their costs with AI and still bill top dollar there will be a rude awakening just around the corner.
"Consultancies will try to swing the billing justification away from total hours worked to value generated but will their customers be convinced?"
There are of course some "share the benefits" lawyers, and they're almost entirely the low rent vermin offering no win, no fee, or "claims assistance". In the big grown up world however, commercial law firms have been trying this for at least two decades without any luck (since the days I worked for a big London law outfit). Whilst there must be some odd datapoints that disprove my absolutism, I can't think of any serious commercial law client that's "sharing the value benefits" with their law firm. And if the law firm do have that discussion, the retort from the in house legal team will be "that sounds very promising...of course you'll share the losses on all deals that go south?" It's at exactly that point that the value generated argument gets put studiously back into its velvet lined box, and the law firm walk out as primly and tight stepped as if they were trying to hold in a week's loose stools, and go back to billable hours*.
* As in excessively padded, let's hope we don't get challenged billable hours.
The word advice in that comment covers a scandalous level of deception.
In seeking "efficiency", a series of right-wing governments in Australia fired much of the public service, only to replace the advice public servant experts used to give with advice provided by for-profit consultancies, of which KPMG were one. Many of these consultancies happened to be donors to the right wing parties.
These consultancies also had large numbers of corporate clients who used them to minimise their tax liabilities. When the government introduced new tax regulations, they included, on consultants' advice, loopholes which KPMG had already advised their clients to structure their finances to exploit.
Needless to say, Australian governments don't use KPMG any more.
“ However early experiments produced "really scary" results including the discovery of a single document on KPMG servers that listed thousands of employees' credit card numbers.
"That absolutely scared the pants off me," he said. KPMG therefore stopped its experiments and blocked ChatGPT while it assessed the risks AI posed.‘
So instead of improving security and data protections, they stopped using the tool that had discovered the insecure data…
