The worst part of having a small sample size is not knowing. If all of your samples have good performance, you don't know the likelihood of failure. If you have a failure, you don't know the percentage of good samples you will expect to have in the future.

Basically, low sample sizes breeds pessimism because of human nature's tendency to predict the future based off of incomplete facts.

However, many times we do not have the luxury of testing many samples, so what can we do? Don't predict the future.