Beyond Algorithms: Can AI Truly Understand the Human Mind?

I am writing this piece while sitting at a quiet café in one of the rural towns of beautiful Portugal, the world moving subtly around me—a few people passing by, the trees swaying with the light breeze. Across the table, a friend sitting, sharing the silence, each of us picking up on the small, a raised eyebrow or a soft smile, perhaps unspoken cues that make up human connection. This moment reminds me of how Theory of Mind operates in our everyday lives. It’s not just about understanding words but sensing the unspoken—the way someone might look out the window or let a pause linger, hinting at what they feel without saying it. As I sit here, I wonder: can AI ever truly grasp these nuances? Could it understand what this moment feels like? And if so, could it engage in the same rich, unspoken ways that make human interaction feel genuine and profound?

These questions lie at the heart of current research into Theory of Mind (ToM), a cognitive ability essential to both social cognition, human psychology and the next frontier in artificial intelligence.

In my recent article, Anthropomorphizing AI: The Next Frontier in Our 18,000-Year Journey with Technology, I explored how our long relationship with technology has led us to imbue machines with human-like qualities. Yet as AI evolves, it’s not just about performing tasks but about understanding our social cognition. At the core of this journey is ToM. Let’s explore it together:

Defining Theory of Mind and Its Role in Human Interaction

Theory of Mind is the ability to attribute mental states—beliefs, intentions, desires, and emotions—to ourselves and others. This capacity enables us to recognize that others may have perspectives and knowledge different from our own, a skill crucial for understanding and predicting human behavior. From early childhood, ToM develops as a foundational element of social cognition and interaction, helping individuals interpret the world through the lens of others’ minds. 

ToM in Practice: “The Room with the Open Window”

In the story below, The Room with the Open Window, we dive into four key aspects of ToM—hinting, false belief, irony, and faux pas detection—capturing both the subtleties of human interaction and the challenges AI faces in achieving genuine understanding.

The Room with the Open Window

Lea, sitting in a small, quiet room with a single window that opened onto a city street. Outside, cars moved slowly through the evening mist, their lights glowing softly in the early dusk. Inside, Lea sat with a friend across from her, a cup of tea between them, steaming gently in the cool air. Each interaction that follows explores different ToM tasks:

1- Hinting Task

Lea quietly mentions, “It’s quite cold in here,” indirectly hinting at the open window. Her friend understands the hint and closes it without needing a direct request.

2- False Belief Task

Lea’s child places a marble in a box and leaves. While the child is away, Lea moves the marble to a drawer. When the child returns, they believe the marble is still in the box. This captures a classic “false belief” moment, where a person’s belief differs from reality. 

This reminded Lea about the “Sally-Anne” psychological test, which demonstrates the ability to recognize that others can hold false beliefs. She noted that children with autism often find this challenging, sometimes due to distractions in the setup or interpreting repeated questions as prompts to change their answers.

3- Irony Comprehension

Lea’s friend says, “Great job!” after the child drops the marble under the table. While the words are positive, the tone holds playful irony, which the child misses but Lea picks up on.

4- Faux Pas Detection

Lea’s friend notices her looking a bit tired and casually remarks, “ou look really worn out today.” He means it as friendly small talk, unaware that she was actually up all night caring for a sick family member. Lea feels a moment of discomfort, but she understands that her friend didn’t intend any harm—it was just a well-meaning comment that missed the mark. This highlights a subtle social slip or “faux pas.”

Historical Context: Themistocles’s Use of Theory of Mind at Salamis Battle

In “The Room with the Open Window,” we see how subtle human interactions involving hinting, false belief, irony, and faux pas detection reveal the intricacies of ToM. Such social cognition appears not only in daily interactions but also resonate on larger stages—such as the battlefield. The intersection of cognitive insights in warfare, historical strategy, and the evolution of AI highlights a remarkable journey toward understanding and emulating human decision-making.

Take, for instance, Themistocles, often hailed as the “Father of Naval Warfare,” at Battle of Salamis (480 BC) skillfully employed cognitive ToM—the ability to infer beliefs, desires, and intentions or “read the mind” of his opponents— by understanding and shaping the Persian fleet’s perception of the Greek forces. Anticipating that Xerxes and his commanders saw the Greeks as vulnerable, Themistocles sent a deceptive message, suggesting that the Greek fleet was in disarray and might flee. This calculated move exploited the Persians’ belief in their own naval supremacy and prompted them to attack. When the Persians advanced into the narrow straits of Salamis, they were met by the well-prepared Greek forces who had strategically positioned themselves to take advantage of the confined space, leading to a decisive Greek victory. Here, Themistocles demonstrated a masterful understanding of human psychology that is central to ToM, impacting the outcome of this historic conflict. Indeed, Themistocles’s approach wasn’t just a demonstration of military skill; it was a profound exercise in understanding human motivations and leveraging them to achieve a decisive advantage. 

Similarly, at the Battle of Hattin (1187 AD), the Crusaders were lured into a disadvantageous position, but this time by exploiting affective ToM, which involves understanding and influencing others’ emotions and motivations. The besieging of city called Tiberias was intended to provoke an emotional response from the Crusaders, compelling them to relieve the city despite the strategic risk. The Crusaders’ loyalty and urgency to protect Tiberias led them across the arid plains, where they were drawn further from water sources. By the time they reached Hattin, they were exhausted, dehydrated, and demoralized, facing a well-prepared force. The Crusader leaders, driven by affective motivations and the emotional weight of defending their people, had walked into a trap. This manipulation of their affective responses created the psychological strain necessary to weaken their resolve and coordination, ultimately leading to their defeat. Furthermore, this foresight in targeting their resources, especially water, created a mental strain on the Crusaders. This type of cognitive pressure aligns with modern cognitive warfare tactics accompanied by AI where adversaries exploit psychological vulnerabilities to weaken resolve and disrupt coordinated decision-making and to also distort signals to force adversaries into unfavorable actions based on distorted preferences.

In comparison to historical strategies such as Themistocles during the Battle of Salamis—where understanding the opponent’s motivations and intentions was crucial—these AI systems struggle to replicate such insights effectively. Themistocles anticipated the Persian commanders’ expectations and shaped their actions through his calculated misdirections, capitalizing on his understanding of human nature. In contrast, modern AI systems in warfare struggle to reach this level of insight. While such systems moved from traditional AI of predefined rules and excelled in processing data and identifying patterns, they still lack higher-level cognitive capabilities—such as commonsense reasoning and adaptive interpretation of human psychology and intentions, which can falter in the unpredictability and “fog of war” scenarios. Reaching a true Theory of Mind, thus, needs a contextual adaptability that human cognition naturally employs in unpredictable situations. 

Exploring Theory of Mind in AI Systems

Today, as AI systems become more sophisticated, researchers are exploring whether machines can replicate this essential human skill. Large language models, like GPT-4 and LLaMA, can perform impressively on ToM-inspired tasks by simulating responses that appear socially aware. The study by Strachan et al. shows that models like GPT-4 performed at or above human levels in certain Theory of Mind (ToM) tasks, such as identifying indirect requests, understanding false beliefs, and interpreting irony. For example, GPT-4 displayed a sophisticated ability to recognize when someone might make an indirect request, such as subtly asking for a window to be opened on a hot day, by interpreting the requester’s beliefs and desires. 

Challenges in AI Theory of Mind: The Illusion of ToM in AI

However, while these models can mimic certain ToM-like behaviors, they often lack a true, conscious and phenomenological understanding of human mental states, responding instead based on patterns and probabilities.

Indeed, AI still struggles with genuine human-level social cognition including ToM, especially in tasks that require deep contextual understanding, like faux pas detection. The same study by Strachan et al. reveals that models like GPT-4 sometimes avoid making inferences in ambiguous situations, likely due to programmed conservatism designed to prevent overconfident responses. This “hyperconservatism” leads to cautious, non-committal answers when faced with social ambiguity, whereas humans tend to take calculated risks in interpretation.

This cautious approach can make AI seem less human or empathetic, as it often fails to engage in the same spontaneous, intuition-driven judgments that people use to navigate social situations. Instead, it creates an illusion of ToM operating through pattern recognition, responding based on statistical probabilities rather than an understanding of context or emotions. In philosophical terms, this contrast is often referred to as “unconscious AI” versus “Phenomenological AI”: the former mimics intelligent behavior without true consciousness, while the latter would theoretically involve actual phenomenological awareness —a feat that remains hard to achieve today and maybe unattainable in the future.

Bridging the Gap: Generative AI and Theory of Mind

Our research labs at Kiangle.ai is leading in this research and now creating AI systems capable of simulating in real-time dynamic interaction and adaptive decision-making. This approach goes beyond isolated actions by enabling agents to actively update their understanding of complex, evolving situations and make real-time decisions in fluid, uncertain conditions. This approach goes beyond isolated actions by enabling agents to actively update their understanding of complex, evolving situations. For instance, in command and control (C2) operations, these agents simulate multiple courses of action (COA), adjusting for real-time changes and adapting their recommendations based on shifting conditions. By using causal models, these agents also allow AI to align with a commander’s intent, explore alternative hypotheses, and anticipate the implications of each decision and scenario.

Besides, these generative agent models would have a “selective misinformation” trait, which is the agent’s ability to decrease the mutual information between its true intentions (or preferences) and the signals it sends to adversaries. Agents are also equipped with recursive Theory of Mind (ToM) to engage in intentional message distortion to influence another adversaries beliefs and actions. They learn to selectively reveal or conceal information to manipulate outcomes in their favor leading adversaries to make decisions based on incomplete or skewed data and get influenced by deceptive signals.

Despite these advancements, AI’s grasp of human cognition remains limited compared to the intuitive, metacognitive abilities that characterize human decision-making in complex systems, where traits like real-time adaptability and metacognition are essential.

Conclusion: Future Direction

In essence, while cognitively-inspired AI shows promise, achieving human-level ToM remains a distant goal. Integrating AI with human expertise is vital to overcome its current limitations. In complex systems like financial markets, healthcare, and climate modeling, warfare, food security involves interdependent factors, such as agriculture, logistics, weather patterns, economic policies, and social behaviors, all of which can influence and disrupt the system unpredictably. Thus, decision intelligence demands cognitive capacities that neither AI nor humans can handle alone. A 21st-century Saladin cannot single-handedly manage the complexities of modern conflicts.

Human interactions thrive on imperfections, spontaneity, and vulnerability—qualities that AI currently lacks. If we want AI to integrate seamlessly into complex social or organizational settings, a bit of “human messiness”—allowing for missteps and organic learning—is essential. In short, by embracing chaos and inefficiency as part of ToM development, we can create AI that respects not only data processing but also the human lived phenomenological experience, thus, allowing it to “understand” in a genuinely human way, rather than merely imitating it.

Back to the question: Can AI truly understand the human mind? Social cognition is just one of many aspects required to fully grasp the complexity of our minds, and there’s still a long way to go.

This is part of my Safeguarding Against Cognitive Influence of AI Newsletter. If you are interested in this topic and wants to receive future posts, you can subscribe below:

References:

1. Hoover Institution. (n.d.). Hattin. Retrieved from https://www.hoover.org/research/hattin
2. Strachan, J. W. A., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., & Manzi, G. (2024). Testing theory of mind in large language models and humans. Nature Human Behaviour. https://doi.org/10.1038/s41562-024-01882-z
3. Lantern Studios. (n.d.). The History of AI: From Rules-Based Algorithms to Generative Models. Retrieved from https://lanternstudios.com/insights/blog/the-history-of-ai-from-rules-based-algorithms-to-generative-models/
4. NATO. (n.d.). Cognitive Warfare: Strengthening and Defending the Mind. Retrieved from https://www.act.nato.int/article/cognitive-warfare-strengthening-and-defending-the-mind/
5. Understanding Autism and Theory of Mind: The Sally-Anne Test. (2016). Centre for Education and Youth. Retrieved from https://cfey.org/2016/07/understanding-autism-theory-mind-sally-anne-test/#:~:text=Sally%20returns%20to%20the%20room,participant%20(and%20Anne)%20know.
6. Hare, B. (n.d.). From Hominoid to Hominid Mind: What Changed and Why?. Department of Evolutionary Anthropology and Center for Cognitive Neuroscience, Duke University, Durham, NC.
7. Unknown Author. YouTube. Retrieved from https://www.youtube.com/watch?v=ndXuRoJd6hQ
8. Elsevier. Article on cognitive and social theories related to AI. Retrieved from https://www.sciencedirect.com/science/article/abs/pii/S1364661323001687
9. Yoksis Bilkent University. (n.d.). PDF Document on Theory of Mind. Retrieved from http://yoksis.bilkent.edu.tr/pdf/files/16344.pdf
10. A (Dis-)information Theory of Revealed and Unrevealed Preferences: Emerging Deception and Skepticism via Theory of Mind.
11. Themistocles: The Father of Naval Warfare https://cimsec.org/themistocles-father-naval-warfare

---