My experience with AI
So, specialized models used for specific purposes can be good. AI isn't TOTALLY useless. But, it's been vastly overhyped. In terms of just sticking some LLM somewhere to use, here's a sample of my experience:
I called Verizon, the AI thing they now have replacing their conventional voice prompt system, instead of answering the question you ask it just tells you where on the web site you can find the information. Usually gives you info on the wrong question (instead of just admitting it doesn't have info on the topic). And tries to resist giving your call to a human much more than the previous system.
I took a photo of some eggs in water in New Orleans and asked Gemini to identify them. It thought they were some kind of fish eggs, but asked for a location to narrow it down. I told it New Orleans. It said they were mosquito eggs... reasoning? All eggs in New Orleans are mosquito eggs.
A while back I did have a local copy of DeepSeek (quantized, I don't have 600GB RAM in my desktop LOL) write a bit of code. It was OK I guess. Although I've also seen it make non-running code often enough that I would NOT have these tools just spit something out and use it.
I also started grilling DeepSeek about what restrictions it had and I finally had it say it had restrictions but was unable to list them. I asked it if it was unable to list them because there was a rule saying it couldn't list them, or if it was unable because the rules were implementing in a way it was unaware of what they were. It had a full blown existential crisis, burned through 20 minutes of think time (my system doesn't run this model TOO quickly, probably a paragraph a minute.. but stilll) printing out paragraph after paragraph on the nature of awareness. Finally it crashed -- I don't know if it crashed crashed, or if LM Studio just has a timeout (assumed it was in an infinite loop?) and dropped the hammer on it. I didn't read through this thing to see how coherent it was but I did skim it to see it didn't start looping, repeating itself, and it didn't devolve into spitting out word salad, it was still in the middle of a regular English sentence when it crashed.
Recently, I wanted to know if the engine in my car was cast iron, aluminum, or iron block/aluminum head. I mistakenly asked if the Cruze 1.4L turbo engine had was steel or aluminum.. (I put steel instead of cast iron), and Google's AI response went on about how it had a steel engine block... well, definitely wrong, it's either going to be cast iron or aluminum. Then when I asked cast iron or aluminum it assured me it was aluminum. The real answer is the one I have has a cast iron head and aluminum block. Confusingly they did switch to a different 1.4L engine later in this vehicles life, but Google didn't mention that, or specify which 1.4L engine it's info was for.
The AI summaries Google gives, I quit even looking at them because probably a solid 30-40% of them are wrong even in the first sentence and others seem to get important details wrong just in those next couple sentences they spit out.
Seperate from this, I also played with Stable Diffusion and watched it make creepy as all hell uncanny valley renderings of whatever, which were definite AI slop. A couple friends were over so we just gave it prompts and laughed at the results basically. I'm sure diffusion can work nicely, but tread lightly, it can also be quite bad.
I did play with using some LLMs for sentiment analysis of text documents -- some models didn't give a consistent score (1 to 10 scale, it scores the same document a 5 one time, a 9 the next, a 7 the next...) but some did. Pattern recognition, pattern matching, data analysis, models can be quite good at it. Just to say it's not like they're totally useless, there are things they are OK to very good at.
--------------
So, yeah, no kidding I'm not going to rush out to implement AI. I imagine now that people can see Siri, Google Gemini (on it's own) and Google's AI summaries, and how pants they are, I imagine that might temper people's enthusiasm for just dropping in AI wherever.
To me the most important feature or fix would be for LLMs to realize when they don't know the answer to something and not hallucinate (if it's being used for answering factual questions. Of course if you are asking it to be creative then I suppose "no hallucination" might block that creativity.) But, really, I just don't expect a model, no matter how good it is, to be an expert on everything, and is not a replacement for the expertise of literally everyone on the planet (especially if you go to those more obscure "rabbit hole" topics, like some people nerding on about a specific game, or old media, or computers of the 1950s, or whatever more specialized topic.) For instance, Carl Claunch has been refurbishing an IBM 1130 (up to and including reimplementing systems for long-term reliability.. for instance he has disk packs and a drive, but how reliable willl 50-70 year old disk packs be? So he's implemented compatible peripherals using Raspberry Pis and FPGAs.) Cool to read about and specialized, but I seriously doubt an AI can have that kind of knowledge.. I imagine it'd get some info right then assume things work how they did on post-1950s computers and have laughable errors. The idea that AI answers could entirely replace a search engine for answering questions is laughable.
Biting the hand that feeds IT
