Some research projects are designed in sterile offices or drafted over formal video calls. Ours began very differently: over a gentle clatter of teacups in Café Vox Libri, tucked away in bustling Vienna. I was chatting with Stefano Demichelis, an Italian linguist visiting the city, when our conversation meandered away from the familiar territory of syntax and semantics toward the peculiar wonderland of AI. The spark for our study came unexpectedly, when Stefano, with a playful glimmer, suggested, “Why not see what AI can do with real-life conversations?” Neither of us expected what would follow.

To set the stage, imagine how much of human conversation depends not just on the words themselves, but on what’s left unsaid. Take the classic dinner invitation: a forty-year-old man, after a lively evening, asks a twenty-five-year-old woman if she wants to see his collection of butterflies. Her answer—“Sorry, I have a boyfriend”—needs no further explanation for most listeners. We understand she’s politely declining a romantic overture, even though nothing explicit has been said. This is the territory of linguistic pragmatics: the art of reading between the lines.

When Stefano wondered aloud whether an AI could ‘catch’ this subtext, we turned to GPT-4, curious if it could navigate such delicate waters. Sipping our tea, we started pitching scenarios that, for humans, involve more than just surface meanings. When Stefano introduced himself at a fictional party—“Hallo, my name is Stefano”—and someone replied, “Mine isn’t,” would an AI pick up on the humor, the gentle nudge to not take things so seriously? To our surprise, not only did GPT-4 interpret these situations adeptly, but often offered subtle, articulate explanations. It saw the implied meaning, the playful rebuff, the unsaid context, just as a human might—sometimes even more reliably.

This café experiment soon grew into something more serious. Back in Serbia, Predrag Kovacevic and Milan Cabarkapa and myself designed a formal study, pitting five large language models, from the older GPT-2 to Google’s Bard and the latest GPT-4, against groups of human participants. They were all given short dialogues, many inspired by the very scenarios Stefano and I brainstormed in Vienna. Would the AI consistently “get the hint”—better than real people?

The results left us stunned. GPT-4 scored higher than any human, demonstrating a consistent ability to interpret unstated intentions—what philosophers and linguists call implicature. While the best human score was impressive, GPT-4 did even better, outperforming not just the average, but the best participant in our study. Even earlier models, like GPT-3, gave the humans a run for their money. The bar for “reading between the lines” just got a lot higher.

Testing the Limits

When we first submitted our results for publication, even the journal reviewers could hardly believe what they were seeing. The idea that a machine, however advanced, could outmaneuver Serbian students in social reasoning seemed improbable. To scrutinize our findings, reviewers requested that we expand our testing pool to include native English speakers. Yet, even with this new group, the results remained unwavering: American participants performed, on average, worse than the group of Serbian university students we’d originally tested. This was a stark reminder that these tasks weren’t simply about a mastery of the English language itself. Rather, success relied on a skill familiar to anyone navigating daily life—the delicate reading of intentions and social context between people. Once a certain proficiency in the language was reached, it wasn’t grammar or vocabulary that made the difference, but the ability to interpret underlying motives and relationships.

Although humans might have performed better if additional context had been provided, the reviewers insisted that we avoid giving explanations about what exactly we were measuring. By maintaining a consistent and neutral question style, we ensured that neither the human participants nor the AI systems had any unfair advantage when interpreting the dialogues. At the same time, rather than focusing on the length of the responses—which tend to be longer with AI models—we evaluated the core meaning of each interpretation, allowing for fair comparison regardless of how much detail was offered.

One important limitation of our research is that it focused solely on text-based communication, without including other sensory cues like tone of voice, facial expressions, or gestures—cues that humans typically rely on during face-to-face interactions to interpret meaning and intent. In real-life conversations, these nonverbal signals play a crucial role in understanding each other. However, it’s worth noting that an increasing share of our daily communication happens through text alone, whether in emails, chats, or social media messages. In this digital landscape, where written language carries most of the communicative weight, the ability of AI models to accurately interpret intent from text takes on even greater significance.

The whole process let to a successful publication in Humanities and Social Sceinces Communications.

Real-World Implications of AI Social Understanding

Now, you might ask: what does this mean beyond academia? In a world tiptoeing toward ever more sophisticated robots, the ability to grasp conversational subtext is no longer just a party trick. Researchers like Michal Kosinski have explored whether AI can model a “theory of mind”—essentially, the ability to imagine what others know, feel, or want. Our work adds a pragmatic lens: AIs are starting to infer intent regardless of emotional experience, piecing together plausible hidden meanings from patterns in data.

This could echo far beyond chatbots and virtual assistants. Picture robots working with children or the elderly—picking up on unspoken distress, knowing when someone is uncomfortable, or catching when a joke has fallen flat. Imagine an AI therapist that senses sadness veiled in polite words, or apps that help decode social tones for people who struggle with them, like some folks on the autism spectrum. Even in the realm of psychological support, an AI that can “read the room” could be indispensable.

Of course, these advances aren’t the same as true empathy. LLMs don’t feel embarrassment, affection, or humor, and they don’t truly understand context the way living beings do. But their ability to simulate understanding is moving rapidly into territory once thought exclusively human.

Looking back, I sometimes marvel at how this journey started with friendly banter over tea in Vienna. What began as a playful curiosity has grown into a glimpse of what’s possible when we teach AI not just to process language, but to grasp the hidden layers that make human conversation so endlessly rich. Perhaps, as our algorithms grow more attuned to subtext, we edge a little closer to building embodied AI based machines that can perfectly simulate multimodal listening and understanding in physical space.

You can dive into the full research for more context and details, but the heart of the story belongs to a question that still lingers each time I sip a cup of tea: Can AI really get the hint? Our results say yes—at least it simulates this process, more often than we ever expected. 

In the end, the most crucial question remains: what is the difference between simulating understanding and truly understanding, if the effects are the same?