Prompt engineering is a task best left to AI models Large language models have given rise to the dark art of prompt engineering – a process for composing system instructions that elicit better chatbot responses. As noted in a recent research paper, "The Unreasonable Effectiveness of Eccentric Automatic Prompts" by Rick Battle and Teja Gollapudi from Broadcom's VMware, seemingly …
This post has been deleted by its author
But... but... we were told that generative AI was going to create as many jobs as it destroyed, opening up exciting new career paths like "Prompt Engineer".
Oops.
Seriously, when I first heard that term, my reaction was that this was likely to be the next pseudo-profession full of bullshitting YouTubers passing themselves off as experts, followed by further wannabe "experts", all the way down to customers driven by AI FOMO, dazzled by fake "expertise" consisting- like search engine optimisation- of little more than memorisation of ephemeral and shallow tips and tricks masquerading as a science.
A bandwagon you could see countless people jumping on to under the mistaken impression that "prompt engineer" was ever going to be a career with a long term future. When in reality, it's not just that any "expertise" they're going to accrue in how gaming the current generation of LLMs is likely to be worthless in a few years time, let alone something worth building on. (*) It's that the entire concept of a "prompt engineer" itself was so obviously little more than a reflection of the current- but ephemeral- state of the art, and likely to be rendered irrelevant as things moved on.
So yeah, I could have told you that this would happen. But I'll admit that I never realised it would happen *quite* so quickly.
(*) Rather like someone who knows that if you hit their current car in a particular place or jog the accelerator it gets around a particular problem with the engine, but doesn't really understand the underlying mechanics of why that is, or how the engine works. There's nothing to build on, and all that "knowledge" will be irrelevant when that car gets scrapped and replaced by another with completely different foibles.
You raise a valid concern about the hype surrounding new career paths like "Prompt Engineer" in the context of generative AI. It's important to acknowledge the transient nature of such roles and the potential for shallow expertise to become quickly outdated. Incorporating SEO services is crucial for navigating the dynamic landscape of online visibility and ranking. It's essential to integrate SEO strategies into our approach to ensure sustained growth and relevance in digital spaces. This highlights the need for a deeper understanding of underlying principles rather than relying solely on ephemeral tips and tricks. Building sustainable careers requires a focus on fundamental skills adaptable across evolving technologies.
Obviously, Battle isn't a trekkie. If he was, the result might have been relatively obvious, considering the quantity of Star Trek material that the LLM had probably digested while training.
The source of most of the apparently anomalous output of LLMs is most likely bizarre biases in the online cultural data set used for training (see 'social' media for examples). That's primarily due to [a] the dominance of "entertainment" as a motive and [b] the prevalence of everyone with a momentary idea rushing to publish it to the world whether or not thousands have already done so. Given that the LLM understands absolutely nothing but merely assembles a string of tokens based on probabilities inherent in its training, the resulting biases inevitably emerge in its output.
This is most likely not purely a technology issue, so it would be informative to examine LLM behaviour in terms of comparisons between training data, queries and output from a psychological/cultural perspective.
> Given that the LLM understands absolutely nothing but merely assembles a string of tokens based on probabilities inherent in its training, the resulting biases inevitably emerge in its output
I wish more people could understand the above. So many idiots seem to think that AI is some kind of magic, or worse, actually intelligent.
But, it just outputs anything that might plausibly occur in its input data, based on the supplied context. It is incapable of reasoning or inventing anything novel, it just spews a mangled mish-mash of what went before.
It is impossible to examine the reasoning behind an "AI" decision, because there is no reasoning. It's damned lies, statistics, and logistical regression.
No handbags necessary. We have an isolated instance of ChatGPT 3.5 inside our company network that has been trained on a very limited quantity of documentation, mostly that on the intranet.
Quite a bit of that is out of date or obsolete, long forgotten and no longer linked pages.
Suffice to say the output, what little it produces, can only be reflective of the content that it's been fed.
I'd be very surprised if it has *only* been trained on your internal documentation. Unless there really is so much of it that it competes with the entire internet in terms of english language examples.
No, more likely, it has been re-trained via transfer learning, so your documentation is just fed as an additional input to modify its existing weights.
If so it could still exhibit many of the the same biases as the public instance, and could still spew gibberish from the Internet instead of gibberish from your documentation.
Leaving aside other humans, it's impossible to truly explain a lot of what happens in my own head. Anyone who's read Kahneman's Thinking Fast and Slow etc. should recognise that much decision making is not Type 2 conscious thinking, it's happening via a high-speed Type 1 black-box association machine, with any required explanations being invented and retrofitted in retrospect.
...of hearing about "prompt 'engineering'", whatever the heck that is. There is no engineering involved. From what I can see, it simply means "writing grammatically correct English," because of course a chatbot will respond better to "Please do..." than "yO, GeePeeTee, cAin't yOu gimme sum code?"
From the paper: "[a]lthough we aspired to assess widely recognized commercial models such as GPT-3.5/4,Gemini, Claude, etc., conducting experiments involving 12,000 requests per model was deemed financially prohibitive [...]. Consequently, we opted to utilize models hosted by VMware NLP Lab’s LLM API."
So the results may not reflect the performance of the models most folks use. Indeed the authors found that "As evidenced in the subsequent sections, certain overarching patterns become apparent; however, they do not universally apply to each model across all prompting strategies. We will explicitly illustrate that there is no straightforward universal prompt snippet that can be added to optimize any given model’s performance."
This strongly suggests that there's no underlying rationale for the optimisation of prompts, leading to (my) conclusion that there's no actual (even artificial) intelligence present.
Right on! I also love that they have a whole Section, "5 THE REPRODUCIBILITY PROBLEM", where they state:
"our findings exhibit significant discrepancies from the published performance scores [which] underscores a broader issue of reproducibility that has long existed inside the machine learning community".
That, and the C.5 NoQ=10 System Message that includes "The solution to the equation 2x + 3 = 7 is x = 4", suggest the tech is as reliable (and entertaining) as a fire engine operated by circus clowns!
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
