The human factor
Since commercial air traffic became a thing in the late 1940s and 50s, the industry had to deal with countless teething issues. Early planes didn't have redundant control systems, or sensors to measure almost every conceivable parameter, whether for consumption by the pilot or for the FDR. Pilots also didn't have to file a flight plan, or stick to a specific route. That led to an infamous 1950s mid-air collision above the Grand Canyon, when a pilot decided to give his passengers a good look of this natural marvel.
Over the past decades, air travel has become immensely more safe. Every incident and every crash led to improvements and more knowledge that improved airplanes, radar systems and so much more. Some lessons were hard-learned, such as TWA800, and the countless crashes due to sudden downdrafts.
What hasn't changed much, however, is the human inside the cockpit. Aside from that today's generations of pilots are unlikely to have served in the air force, getting most of their experience flying above the battlefields of WWII, Korea and Vietnam. Instead it's mostly about the training that these new pilots have received, which unfortunately doesn't always suffice, as was learned with AF447, where the PF (pilot flying) got confused in the dark over the Atlantic Ocean when he got handed back control by the board computer due to frozen pitot tubes returning conflicting readings, managed to yank on the controls a few times, get the airplane into a left-banking, oscillating turn, nearly stalled the airplane a few times, got confused by the stall warnings before getting the airplane into a proper stall and having it drop out of the skies into the ocean.
Such cases of pilots managing to wreck perfectly fine airplanes for no good reason are sadly becoming a large part of today's crashes. In large part this seems to be due to either the pilot becoming confused and losing his sense of orientation, not trusting the instruments, or becoming overly focused on a single, often irrelevant, detail while ignoring the issues that will kill them in a few moments. Like the captain who insisted on debugging the lights for the landing gear while circling around the airport, until his plane ran out of fuel.
Systems like TCAS, ground and stall warning and ILS are there primarily to assist the pilot, but they're there as suggestions, not as commandments, and it has been decided that the pilot ultimately remains in control. As modern day crashes and incidents show, this is both a positive and a negative thing. Unfortunately, both human and machine are still flawed in the end.
There's been a lot of research and studies by NASA and others on cockpit behaviour, which has led to improved use of checklists and much more. Everything from social interactions between the captain and co-pilot, the adherence to protocol and the dealing with unexpected events all can become a single link in the chain that leads to an accident.
Here I find a phrase that's often uttered in professional pilot circles quite useful when thinking about the right thing to do in a situation as a pilot: 'How would this look in the NTSB report?'