Non open source "must be assumed to be dodgy"
I sure hope you don't mean to imply that open source software can be assumed to be not dodgy. That didn't work out too well for people relying on e.g. openssl, which in the past year or two was found to have some bugs that were just as old as the bugs in this research software - despite being used by FAR more people and being FAR more critical to get right. Yeah, openssl code was rather messy, but research software is generally the ugliest code you've ever seen.
There are only two ways to validate the output of research software like this. One, have a second version of it developed completely independently to act as a check - but who wants to "waste resources" on writing and maintaining it? Two, have someone occasionally work out the results "by hand" to check. I realize some may object it say "what if it needs a million calculations, it would take someone a lifetime to do that" but you can still do it with computers, you just need to break it down into its components and do the calculations separately piece by piece without referring to the standard research software at all.
Even these aren't foolproof - what if the method is based on a paper someone wrote 20 years ago and it turns out there was an error in the paper that no one ever caught? In such a case the software would be "right" in terms of following the formula exactly, and separately developed software or calculations performed manually would show a matching - but wrong - result. Science is hard.
Biting the hand that feeds IT
