It's a psychology thing there.  Your brain doesn't normally look at a whole face, just a few details.  Look at mr. calm, he sorta looks messed up in proportions on his face, when you back away to where your brain only sees a few key details like his eye shape, mouth and forehead, it can't accept that it is a picture of a calm man, but that of an angry one.  It is the reverse with angry guy, the details that make him look angry get lost at a distance, while those that make him look nice, like the edges of his face, stay.