Re: Troubles in Physics
It's true that the results from particle physics are statistical in nature, rather than definitive. But I think they're pretty thorough, looking for consistency good to 5 sigmas. That's pretty damned certain.
The "scandal" in the field involved a more publicity-minded group leader declaring results to the world ahead of their data having achieved 5 sigmas of certainty, trusting that by the time they've completed the work they'd actually have got 5 sigma. This happened in CERN when they were running the SPS looking to confirm the existence of the W & Z vector bosons; Carlos Rubia was the publicity-minded group leader. It paid off - he / they got the Nobel prize, the other more steadied group working with data from the other detector took their time and have been lost to history, even though they published the same result at about the same time. Since then, CERN has insisted on being in control of the release of results, etc, to prevent a similar occurence.
Incidentally, having 2 detectors working and two research groups working independently but supplied by the same accelerator is the way they get "validation by third parties". It's all done at CERN, but the results from one detector and the data processing performed by that team are effectively verifying the results produced by a different team on the other detector. They know that building one accelerator with one detector would mean a lack of verification, which is why they put 2 in.
Pulling signals out of noise is not a magic process, it's a designable, demonstrable, testable, assurable process, to whatever level of certainty one wishes (5 sigmas in the case of particle physics, I understand). The same kind of process is used in probing the structure of the earth using shockwaves from earthquakes; they've been able to determine an awful lot that way, to high degrees of certainty. Radio receivers have become increasingly complex and have been doing the same thing for a long time. No one bats an eyelid at a mobile phone working...
With regard to the Standard Model, they do appear to be somewhat stuck for ideas. It's a model that's grown up around physical measurements. But as for whether it's an adequate model for what Mother Nature actually does is somewhat doubtful... The Wikipedia page on it is straightforward about its frailties. Alternatives are considered - the Standard Model is not the only model people work on, String Theory being another.
The good thing today is that there is an awful lot of data - from accelerators, from astronomy, etc - sitting around and so if some bright spark with an Einstein brain fitted gets their teeth into the problem, there's a lot of accurate experimental data to try fitting to it. My biggest worry is that curating that data is actually a major challenge - there's so much of the raw stuff that they can't even get it out of the detectors without some pre-processing. Someone needs to curate and care for that data, so that it remains available indefinitely waiting for that bright spark to come along. We're not very good at data curation.
NASA nearly came a cropper with the Voyager missions. They've been undoubtedly successful of course. However, a little known activity of the missions was to record the interplanetary magnet fields, things like that; dull, dull data of no apparent value, endless reams of it collected and stored, with no apparent purpose in mind. Until one day (about 10 years ago I recall), someone clever so and so thought of a use for it and requested copies of the data. At this point it was found that, whilst the data had been recorded, much of it was on old reel to reel tape dating back to the 1970s and 1980s, and it had become more or less unplayable. Some tape had become brittle and crumbly. Some had become sticky. Etc. They actually embarked on a massive data recovery exercise, and developed techniques to play these old tapes. The crumbly ones they found they could play once, with dust coming out of the tape pull capstain, which had to be hoovered up as the tape played. The sticky ones they found they could play once, if they put the tape / player into a deep freeze. They actually managed to recover a good proportion of the data, and a pretty good research paper was produced from it. But it was a good lesson in how preemptive data curation is a hell of a lot easier than data recovery off ancient decayed media. I just hope CERN and all the research groups who collect the data take the task seriously.
Oddly enough, the folk who are expert these days in data curation / preservation are accountants in the USA. The data preservation requirements in the Sarbanes Oxley act that was born out of the Enron scandal are pretty tough, with jail terms for failure. There's now a significant industry in accounting data preservation...
Biting the hand that feeds IT
