Re: Any True DB/2 Experts Around ?
> 1.) Query Plan Optimization is a Hard Problem.
Very true. Optimisations are, on the whole, hard problems: just ask any Travelling Salesman.
Which is a damn good reason for being careful about what methods you apply to solve them.
> 2.) For many queries it would be too runtime-expensive to evaluate all possible solutions; heuristics are used.
Yup, such as (in common with other large search space problems) setting up fascinating search structures and pruning them, much (most? all?) of which can be traced back to the AI labs. You know, there was a time when any discussion of heuristics meant you were in a discussion of the AI scene[1][2] - what a shame those two letters are being relegated to just the one thing (yeah, yeah, fighting a losing battle here). Heuristics seen as part of the day-to-day? As noted before "if it works, it isn't AI anymore" - anyway, as you note, heuristics are already used within plan optimisers (just as they are within other code optimisers).
So from that p.o.v. "applying AI (techniques) to the plan optimisation problem" is - in computer terms - a very old strategy; hardly worth mentioning any more, in fact.
> 3.) Maybe AI can add "novel heuristics".
Looking for novel heuristics is a Good Thing - and doing that, plus novel ways of finding novel heuristics - is a good AI topic.
Just - is it the topic that the LLMs[3] are really suited for? Two immediate concerns arise: first, the resource costs[4] to run these beasts.
Second, the (by now) old issue of LLMs happily generating good looking nonsense: that is certainly novel, it also happens to be total twaddle - and applying it within the plan optimiser is rather worse (IMO) than simply (as I half-seriously described it above) applying it to generate junk SQL, from the p.o.v. of your poor beleaguered human DBAs and devs spotting what is going on. Would you really want to trust a heuristic so novel that it was only created in the last few seconds, hasn't been tested out on anything at all yet, may not even ever be run on another query (the "AI" deciding to do something different next time)? Remember, this isn't an AI being run in a lab as part of a concerted research effort to find new heuristics and prove their worth (and limitations), it is something running around doing random things within *your* expensive database.
> 4.) Any query plan, runtime-efficient or not, will produce the same result. AI is not messing with results.
Presumably you mean, any query plan generated from the same input SQL? In which case, *ONLY* if you can demonstrate that the transforms performed are valid in the way they are being performed - and if you are applying some random output from an LLM as one of your heuristics, that is not going to be the case (see above). It may be a sane thing to do, it may not. Prove it. Remember that heuristics include ideas such as "we can prune this from the search because we've seen X" - if the LLM has decided to generate the novel idea that "we can prune because we've seen Y" instead, and it turns out the X case came from a boring long maths-heavy proof back in the 1990 and the Y case just sorta looks like it is following the same pattern...
> 5.) Adding further indices to a DB schema will quickly be a double-edged thing, as index maintenance will also consume serious runtime.
True; picking your indices is almost an art form :-) Not quite sure how that fits into this discussion; are you suggesting that the AI is likely to just keep on adding new indices, because that is a common pattern it has picked up from its training data? Which would be a bit of a reversal, as your previous points seem to be more pro-AI. Dunno; clarification required.
[1] when the idea of applying any "rule of thumb" instead of being rigorous & provable & correct in all your outputs was a bit avant garde - but then, as now, AI trials had a bit of an issue with generating huge (for the time) data and huge searches.
[2] rats - tried G**gling for some references to bolster my memory but half-rembered terms got auto-corrected to hit sponsored links for - nothing I wanted :-(
[3] yes, I know they weren't specifically stated to be LLMs - but (1) the mention of "GenAI" is a pretty big hint and (2) what else is being touted around at the moment? If it really was something else then we'd have been buried under a pile of IBMese marketing speak stressing how much it wasn't just the same as everyone else's
[4] and I'd bet there is someone in IBM following the Sunk Costs argument of "we've built the damn thing, now we have to use it absolutely everywhere"