ChatGPT hasn't been around for long and Nvidia already wants to put a leash on it ChatGPT's testing phases pretty quickly revealed that the OpenAI chatbot and others like it could go off the rails – or "hallucinate" – with enough poking and prodding. OpenAI's sticking plaster was to limit the amount of queries a user could make before the chatbot descended into madness. Now, for businesses perhaps wary of …
It's more like an expert system.
A quick look at the NeMo Guardrails info in NVIDIA's technical blog post shows it's just a framework for writing rules in Colang, a data-flow language NVIDIA invented for this sort of thing. You basically tell it "if the input matches this pattern, do this thing", where "pattern" is a fairly expressive DSL. But there's no magic here, and frankly my initial guess is that if you have programmatic access to a NeMo-protected system (so you can drive a lot of queries) you'd be able to train an adversarial model pretty quickly to simulate its guardrails and then look for holes.
And, of course, it's up to the application developer to design good guardrails in the first place, and that's an area where very few developers are likely to show much expertise.
So this is likely to be used reactively: "Oh, someone just made our bot do a Bad Thing, so write another guardrail to prevent that specific prompt". No magic, no silver bullet.
(That said, a smart organization deploying an LLM-based service would do the adversarial modeling first, and use it GAN-style to create a set of guardrails. That's likely to be more successful than an economically-feasible human-authored set.)
I LIKE TO hallucinate! :(((((((
I had this discussion with someone with a much different circle of friends. I was telling her that I didn't like the side effects of my (medically prescribed) drugs, but could understand that some people might enjoy them. She immediately informed me that there was no "might" about it -- her acquaintances did indeed choose the roller-coaster of hallucination, speeding, fear, and elation.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
