Re: Proof
I thought of a very similar perspective. The perceived existence of an exception demonstrates (proves) the existence of a general rule in one's mental model. (In considering a case exceptional, there must be some rule one considers generally true or being surprised by an outcome implies that accurate prediction was expected.)
Yet it seems that there is some truth to the concept that having known exceptions implies that the rule has been tested enough that the common cases abiding by the rule are actually likely to be common cases (at least within a testing/use context). I.e., having known exceptions implies that the rule has been tested and validated as a general rule (and not just a hypothesis), "proving" the rule.
The concept that exceptions probe the rule also makes sense.
(Even the context of proof/demonstration is significant. Legal or historical proof is not the same as mathematical/logical proof or even scientific/statistical proof.)