Survivorship Bias - Look for what You can't see

Survivorship-Bias describes a statistical error where datasets are not representative of a problem due to omission of data that didn’t make it through the dataset selection process.

Here is an example: During WW2 a British engineer was ordered to engineer better aircraft reinforcements because too many of their fighters didn’t return from their missions. He was instructed to find out in which areas the aircraft where hit the most, so that he could reinforce the proper areas. He came up with this pattern of damage:

Look at this picture. The red dots resemble areas where the aircraft were hit and damaged the most. Where should the armor of the aircraft be reinforced? Obviously in the Areas with the red dots right?

Wrong. Those areas are exactly the spots where, even under heavy fire, the aircraft could sustain damage and still return from their missions. The given dataset creates a false impression of actual fatal damage because the aircraft that did not return were not considered.

The story goes the engineer spotted this logical flaw after his subordinates came to the wrong conclusion. Consequently, he reinforced the aircraft in the areas that weren’t hit.

This cognitive blindspot applies to many different conclusions drawn regulary. It pays to consider what one is not able to see at first glance.