Background: Black Mirror is one of the most famous Netflix series showing cutting-edge technologies and their possible impacts on human life. In the fourth season third episode, “Crocodile”, they showed us a device called recaller. Recaller is a device that enables us to see other people’s memories of episodes of their life (i.e., episodic memory). It has three main structural components: a chip which is put on the subjects’ forehead, gets the subjects’ memories, processes it and transmits the output to a screen component which shows the visual presentation of subjects’ memory in high resolution, and an earphone which allows you to get the sounds in observed memory.
How do we see memories in the brain: One way to see memories is to look at the brain’s processing units, neurons, with a special microscope that shows their activity. To do that, they give particular proteins to neurons, which makes them light up when they are activated. By observing how a mouse’s neurons light up in each behavior, you can associate certain neuronal activity patterns with certain experiences. This basically enables you to read the mouse’s memories. Further studies could use detected neuronal activity to implant a false memory to a mouse! Injecting some light-sensitive material into neurons that encode receiving an electric shock, they made a mouse have a memory of receiving an electric shock in another environment through a flash of light (Liu et al., 2012).
How about watching the memories on a screen?: So, we can see the neuronal activation patterns of some experiences. However, can we actually reconstruct such experiences? Nishimoto et al. (2011) were able to do this. What they did was recording the brain activity of the people in fMRI while they are watching random videos. After training a machine learning algorithm with the videos and associated brain activities, they could reconstruct what people see in the scanner by using their brain activities. Look how similar the constructed versions below:
So, did Nishimoto et al. (2011) do something like recaller?: Not exactly. The main detail about the Nishimoto et al. (2011) study was they could only reconstruct the known scenes. In other words, they knew what brain activity is associated with which video scene. Without such information, such reconstruction would not be possible in their study.
It all comes to the data again: If we can have a big dataset of average brain activation of many people to certain scenes then, we could reconstruct the unknown scenes.
Hardware issues: Also, Nishimoto et al. (2011) used fMRI to record brain activity, which is a huge device where you need to stay still for a long scanning time. In contrast, recaller device is only one chip that is implemented on the forehead easily without any surgery. Alternatives to fMRI might be EEG type of wearable electrodes, which achieved amazing successes in the brain-machine interfaces. However, EEG has very low spatial resolution and using it to decode something as precise as memories might be very hard.
Conclusion: Current research is getting closer and closer to have such a system like recaller. However, it all comes to the engineering to develop a brain imaging device with a high signal to noise ratio, which can be implemented quickly and safely to collect data from humans. Recent products like Neuralink are very promising developments in this direction. Soon enough, we might be able to see someone’s memories.
My video about Neuralink: https://youtu.be/VyRcz7qa_H4
–Edit: a very similar blog post and with more details; https://www.blakeporterneuro.com/is-black-mirrors-recaller-real/