Umm, this is realistic
Actually you might be surprised at how far this has come since the competition. You should go look at the Cyber Grand Challenge site and the results of it. They have some great videos and commentary about the competition and results. The systems involved actually were given only compiled code and no source. They then analyzed the code, developed exploits and defenses, and attacked each other with no human intervention. That is what made it so amazing.
The other part of the story is that the software and environment the systems were working in was intentionally limited and isolated. They ran in a reduced environment that was simpler than the software environment that is found on a typical PC or notebook.
The programs introduced to the systems had been coded with flaws that often were similar to major flaws that have been found in common real world software. In addition to identifying what they were meant to find, the systems also found and exploited errors that the authors did not know were there.
The surprising thing about the results were that many people thought this level of performance was not possible. Earlier rounds of the competition did not show nearly as much promise but the actual competition provided many surprises to many of us in the industry. It is the speed of progress that is part of the surprise in addition to the results.
And yes, the systems were able to look at other systems attacks and use that knowledge to patch their own systems and attack others. All this without software or additional knowledge. This behavior is a common thing in attack/defend style cyber CTF competitions played among humans and it is not surprising that the authors of the CRS knew this and incorporated this behavior too.
There are several companies (for example Trails of Bits, who I am in no way affiliated with but they have a blog post up on the subject called "Automated Code Audit’s First Customer" in which the same principles (even some of the same CRS code) was used on unrestricted real world software for analysis with great success. This stuff is already in the real world and expected to be more mainstream in 2017.
Biting the hand that feeds IT
