Re: "I don't see how this actually protects against IP theft"
"If the user base of an LLM - the buyers of the stolen goods - know that the output can't be relied on, then in theory they'll stop using it. "
Really? And how will the buyers know that they shouldn't rely on it? Most people accept computer output as authoritative --- "Garbage In, Gospel Out" was true 40 years ago. Being able to get a response from a "natural language" prompt only adds a veneer of credibility.