Note: Words highlighted like this are technical terms or other jargon that have been included optionally for those familiar with them or who want to learn such terms. Other key terms may be present, but these can be ignored as desired without any loss of information as you read the article.

Imagine a medical student going to their professor’s office hours to ask for help knowing how to tell if a patient’s MRI brain scan indicates a glioblastoma, a specific type of brain tumor. The student has been struggling to differentiate between scans of this tumor type compared to others and hopes their professor can explain what to look for. The professor instructs the student to look for the shape of a butterfly and to the student’s surprise, it works. This type of tumor does look like a butterfly, and the student now confidently can identify them by looking for this “butterfly sign”. But can such a seemingly silly and subjective method of diagnosis actually work, and if so, how do our brains do it?

Pareidolia 101: “Hey that kind of looks like…”

Diagnosing by lookalike signs can and does work, but to understand how we need to step away from our medical student and zoom out to the level of the brain in general. This phenomenon, called pareidolia, is our tendency to see patterns and meaning in stimuli that does not actually have any [2, 3, 4]. Pareidolia occurs frequently in daily life, especially visual pareidolia, as we often “see” familiar images in regular objects [4] such as a “man in the moon”. If you’ve ever argued over what a cloud looks like, or seen an inkblot, you’ve experienced pareidolia.

Our brains are very good at this pattern recognition, and they ought to be. Our ancestors that were bad at it were far less likely to survive to pass on their genetics. Such evolutionary pressures wired certain brain processes, like facial recognition, to be as rapid as possible [6] – after all, thinking you see a tiger in an empty bush is a much smaller mistake than not thinking you see anything in a tiger filled bush. The visual system of all primates is highly tuned to prioritize seeing faces, with specialized brain regions and networks for the purpose of recognizing them [7]. It’s no wonder then that seeing illusory faces is the most recognized form of pareidolia [3, 4].

Facial Processing: How does your brain see a face at all?

Neuroscience research has found that our visual systems process objects that cause face pareidolia in the many of same ways as we process real faces [2, 8, 9]. The first part of processing a face, whether on a real person or in a pareidolia illusion, is seeing the face at all. Seeing in this initial sense is something your brain is doing as you are reading this article; the light from your screen is hitting light-sensitive cells in the back of your eyes, and these cells are activating and signaling to neurons, the signaling cells of your nervous system. The signals from your eyes are then carried along your optic nerve and routed through your brain’s relay station for sensory signals – the thalamus – before being finally sent to your occipital lobe, where your brain’s main visual processing areas are. In the occipital lobe, signals first go through your primary visual cortex where your brain processes basic features of the information sent by the eyes. After the primary visual cortex has identified elements line lines, movement, and shapes, the signals get sent to more specialized areas within the visual cortex that help process such specifics [10].

Once the visual cortices finish, the processed visual signals are sent along one of two specialized pathways. One pathway, the dorsal visual stream, goes upwards towards the movement and sensory-motor areas of the brain in the parietal lobe [10].
But for recognizing faces, we are interested in the other pathway. The ventral visual stream runs from the visual areas in the occipital lobe to the underside of the temporal lobes. This network of brain areas is specialized for object recognition; here your brain integrates processed visual information with your memories and knowledge about the world to recognize what the shapes and patterns your eyes picked up actually mean [10]. For example, a specialized area within the ventral stream known as the fusiform face area plays a critical role in identifying faces from other objects. This brain region takes in bottom-up sensory information from the ventral visual stream as well as top-down signals of cognitive control from the frontal lobes to rapidly differentiate between faces and non-faces [11].

Face Pareidolia In The Brain: How does your brain see fake faces and how do we know?

The fusiform face area is not part of processing real faces, however; it is also similarly involved in the pareidolia experience of seeing faces in objects [6]. Our brains pay extra attention to objects that trigger face pareidolia, similarly to how we do with faces [9]. Objects that trigger face pareidolia and seem to have an emotional facial expression even activate the same brain networks involved in processing real facial expressions [8]. But how do researchers know this?

One neuroscience research tool used to investigate the functions and activity of brain regions is functional magnetic resonance imaging, or fMRI. The “MRI” part of fMRI uses highly powerful electromagnets and detectors to take pictures of very thin layers of an object like a body. The MRI can “see through” solid structures like bone so is a powerful tool for looking at objects like brains that are not generally visible and cannot be taken out for a better look without major problems. MRIs make 3d scans from 2d images analogously to how a 3d printer can create a 3d object from printing flat layers. By stacking very thin layers one on top of the other, the printer can create a complex structure; similarly, the MRI takes images of flat “slices” of the brain that then are ordered to create a 3d view. But unlike the 3d printer, the MRI is not limited to only stacking these layers bottom to top; it can take images top to bottom, front to back, and side on profile perspectives. Just like in other forms of photography, different angles offer different perspectives which can be of greater use in different situations. Think of a cameraman filming a car race; for most of the track, a top-down view offers the clearest perspective on the positions of the racers, but at the finish a side on angle viewed right down the finish line can sometimes better display who crosses first.   

The “f” in fMRI refers to a specific type of MRI that can interpret activity levels in different parts of the brain by measuring the amount of oxygen delivered by blood flow. The way fMRIs use this blood oxygen level dependent, or BOLD, signal to infer brain activity can be thought of like using traffic cameras in a city to see how much activity is happening in a buildings, blocks, or areas. Traffic cameras won’t be able to look into a hotel to see if it is full of office workers attending a conference, or travelers napping. But traffic cameras can look at how many cars are going to which places, and where those cars are parking vs traveling through. If the cameras see lots of cars are going to a building and dropping people off, you can probably tell that there’s something going on there. This is just like how fMRI’s measure brain activity based on blood oxygen levels; when a brain region is active, the neurons are more active which the fMRI cannot measure, but this requires the neurons to use more oxygen so they end up exchanging more oxygen with blood flow which the fMRI can measure – just like the traffic cameras seeing more traffic dropping off and picking up people. 

As an example, in their 2018 study “Neural mechanisms underlying visual pareidolia processing: An fMRI study”, Akdeniz et al. showed participants pictures of real faces or objects that triggered face pareidolia while recording their brain activity using fMRI. The participants were also shown scrambled versions of the same images to control for the brain activity involved in just seeing an image at all. Using some statistical models, we won’t get into here, the researchers found that the primary motor cortex, a secondary visual cortex area, parts of the ventral stream including the fusiform face area, and parts of the pre-frontal cortices were active in response to seeing both real faces and face-pareidolia inducing objects [2]. This finding in conjunction with similar studies supports the idea that the same brain networks involved in processing faces normally create the phenomenon of facial pareidolia.

Real World Applications of Pareidolia: Putting it all together

Returning to our medical student, we find that recognizing a diognostic sign by looking for a image it looks like may make more sense than it at first seems. By taking advantage of our brain’s tendencies to find patterns in sensory perception, we can use pareidolia as a build in heads up display of sorts. In the world of medical imaging, many pareidolia based “signs” do just that and help clinicians recognize and diagnose based on tumors that look like butterflies, or panda faces, and more [1].