From Cold Brew Coffee in 3 Minutes to Cyber Espionage: The Hidden Power of Ultrasonic Waves

In much the same way that ultrasound revolutionized cold brew coffee this year—by cutting down brewing times from 24 hours to mere minutes through rapid extraction—attackers are leveraging ultrasonic waves to carry out stealthy tracking across devices. For context, ultrasound technology in coffee machines works by breaking down coffee granules faster, creating a brew with the same flavor in a fraction of the time. But while we’re perfecting coffee, attackers are perfecting the art of silent surveillance. From brewing rich flavors to extracting personal data, the power of ultrasound proves versatile—be it for the kitchen or for digital espionage.” Ultrasonic waves, which are sound waves at frequencies higher than human hearing, are commonly used by animals like bats for echolocation. Bats emit these high-frequency sounds and use the echoes that bounce off objects to navigate and hunt.

This silent method of using ultrasonic waves allows attackers to monitor and profile individuals without their knowledge, connecting data across devices like smartphones, smart speakers, TVs, and earbuds. Inaudible to humans, these signals are transmitted through solid surfaces or air to gather information about people’s activities, from what they watch to where they go. This is just one element in a broader approach to tracking behavior—each piece of data, like a dot, connecting to form a detailed profile of an individual’s actions and preferences.

By understanding and predicting these behavioral dots, attackers or organizations seeking to anticipate human behavior can move one step ahead. They mimic human reasoning—biases, heuristics, and all—by incorporating human sciences with AI simulations, which allows them to simulate human-like decision-making and behaviors. Imagine it like a SIMS game, where every action and interaction is tracked and anticipated, creating a virtual representation of a person’s life that can be manipulated or exploited.

Throughout history, the ability to predict human behavior has been a powerful tool for leaders. Political strategists and military commanders have often anticipated actions and reactions, influencing pivotal decisions that shaped the course of history. Whether in ancient empires or modern geopolitical crises, understanding human behavior has been a cornerstone of strategic planning. And now, in the digital age, this same predictive capability has taken on new forms—using ultrasonic waves and advanced AI models to silently track, simulate, and anticipate behavior, both in the physical world and online.

But the power of ultrasonic waves doesn’t stop there. Everyday appliances like printers, coffee machines, lamps, refrigerators, and HVAC systems can also emit ultrasonic waves. These devices, although not involved in cross-device tracking intuitively, play a crucial role in indoor localization. Ultrasonic waves emitted by these appliances create opportunities for accurate, real-time tracking within buildings—mapping movements, identifying locations, and enhancing navigation. This demonstrates the versatility of ultrasonic technology, from connecting devices to optimizing indoor positioning systems.

Examples from Everyday’s Life

Imagine you’re sitting in your cozy living room by the fireplace, wrapped in warmth, feeling under the weather. You just saw a TV ad for a Vitamin C drink, so you grab your tablet device and search for it. But what you don’t know is that ultrasonic waves, hidden in that ad, have linked your devices—the TV, tablet, and phone—all silently talking to each other. Now, advertisers (or attackers) know about your search, and they’ll follow your journey, tracking you even to the store as you pick up that Vitamin C drink.

It can be summarised in these three steps:

1. The Hidden Transmitter: Ultrasonic waves, like a secret handshake, are emitted by a transmitter—an everyday object like a TV in your living room. These signals carry tiny pieces of data—like your phone’s ID or a tracking tag—coded into sound that only your device can “hear.” It’s a bit like how bats use echolocation to find their way in the dark, using sounds that humans can’t detect.
2. The Silent Receiver: Your phone, earbuds, or even a webpage you’ve visited is unknowingly listening. These devices, equipped with microphones or gyroscope sensors, pick up these ultrasonic waves and decode the embedded messages. No permission is needed. The attacker doesn’t need you to grant access to your microphone or location data; they’re simply using sound as their tool.
3. Building a Digital Persona: Over time, these signals help build a map of your behavior. What shows do you watch on TV? What shops do you frequent? Without ever needing to interact directly, the data collected through ultrasonic waves is used to build a profile of you, connecting dots between your devices, your habits, and your preferences.

In this silent exchange, you become part of a vast tracking network—one you didn’t even know existed. The invisible power of ultrasonic waves shifts from nature’s tool for navigation to a tool for quietly monitoring human behavior in a digitally connected world.

Let’s consider another scenario. Imagine sitting at a café, your phone resting on the table while you sip your coffee, unaware of the silent conversation taking place. Ultrasonic signals, invisible and inaudible, are bouncing off the table, injected by an attacker into your phone’s voice assistant. These commands could be as simple as “unlock phone” or as invasive as “read messages,” but you’d never hear them. This creates a two-way interaction—the attacker sends commands like “read my messages,” and your voice assistant audibly responds, while hidden microphones capture the exchange. With continual commands, like “reply” or “forward,” attackers gain deep access to your personal data, all while you remain unaware. This method, initially known as DolphinAttack and evolved into SurfingAttack, which allows two-way interaction between your device and the attackers. hackers to bypass your awareness entirely, gaining access to your data without needing to physically touch your device.

In a similar scenario, with EchoAttack, even your AirPods—those trusted companions for music and podcasts—become vulnerable. Picture yourself at a train stop, enjoying your favorite tunes. What you don’t realize is that while your music plays, ultrasonic waves are bouncing off nearby surfaces, silently sending commands to your earbuds. As they engage your voice assistant, they could be reading your messages or responding to them—without you lifting a finger.

Here’s the reconstructed gym scenario with EchoAttack in more detailed steps:

1. Gym Setting:

• The attacker enters a gym and strategically places a portable gym bag with an ultrasound speaker hidden inside. The victim, working out on a treadmill, wears smart earbuds, e.g., Pixel Buds, that support voice assistants (Siri, Google Assistant, etc.).
• Target Information: The attacker’s AI algorithm leverages knowledge about the reflectors around the victim, such as the gym’s treadmill screens and walls, to calculate the best ultrasound path (indirect or direct) to the victim’s earbuds.

2. Ultrasonic Attack Launch:

• Inaudible Command Transmission: The attacker uses the speaker to transmit inaudible voice commands in ultrasound frequencies. These commands, such as “read my emails or open a voice recording” are picked up by the victim’s earbuds without their awareness. Due to the nonlinear effect of the earbuds’ microphone, the earbuds interpret the ultrasound as valid commands.

3. Feedback and Continuous Monitoring:

• Bluetooth Sniffing: Using Zigbee radios, the attacker sniffs Bluetooth signals emitted between the victim’s earbuds and their phone. This allows the attacker – using an AI decision tree machine learning model – to monitor whether the inaudible commands successfully activated the voice assistant and to adjust the attack strategy if needed (e.g., resend commands).
• Indirect Path Optimization: The attacker may rely on ultrasound reflections from the gym’s reflective surfaces (like the treadmill screens or walls) to increase the signal strength and improve the success rate of the attack.

4. Real-time Sensitive Data Exfiltration:

• The attacker sends repeated commands like “read” or “forward messages” to steal private information (e.g., emails, messages). These responses, which are meant to be audible only to the victim, are captured by hidden microphones placed near the treadmill, unbeknownst to the victim.

5. Extended Attack with Follow-up Commands:

• After successfully extracting messages, the attacker continues to send additional commands, such as “reply” or “delete messages,” to manipulate the victim’s smartphone. If necessary, the attacker can also send harmful commands like “enable airplane mode” or “disable Bluetooth,” potentially disconnecting the victim from the network without their awareness, or “turn on flashlight” to drain the battery quickly.

6. Stealth Maintenance:

• Throughout the process, the attacker remains out of the victim’s line of sight and adjusts the attack based on feedback from Bluetooth sniffing, ensuring that the attack proceeds without alerting the victim.

This scenario illustrates how attackers can exploit the vulnerabilities of smart earbuds and use indirect attack paths in reflective environments, such as a gym, for continuous unauthorized access to personal information. Do you think that if suck attacks can be done on a large scale, forcing massive phones to initiate a call to jam emergency services such as 911 in order to overwhelm emergency service lines and contribute to a denial-of-service-like attack?

Tracking Location:

Besides, ultrasonic beacons can also be used to track location. Attackers place these beacons in public spaces like retail stores. As the victim’s smartphone or earbuds pick up the signals, they reveal the user’s location in real time. For instance, a store might track how often you visit, how long you linger, and what sections of the store you frequent, all without your explicit knowledge. This location data can then be cross-referenced with other signals to paint a more detailed picture of your daily habits. So, while you walk beside a store in a mall equipped with an ultrasonic beacon devices, the system could detect your proximity and offer you, on your mobile, a discount on your favorite product, right on the spot!

Advantages of Ultrasonic Attacks:

• Stealth: The use of inaudible sound waves allows these attacks to operate completely undetected by the user.
• Wide Range of Devices: From smart TVs to earbuds, smartphones, and more, ultrasonic attacks can target a wide variety of connected devices.
• No User Permissions Needed: Many of these attacks don’t require microphone access, which bypasses security restrictions that might otherwise block apps from listening in.
• The surveillance does not rely on GPS or active RF transmissions, making it less susceptible to electronic countermeasures.
• Unlike RFID or GPS-based tracking that requires the adversary to unknowingly carry electronic equipment, this method passively detects the presence of adversaries as they move near the equipped appliances.

“Indeed, even if you turn off location services or disable the microphone on your phone’s apps, ultrasonic waves can still track you and your affinities & preferences!”

Ultrasonic Waves Usage in Robotics & AI – Between Yesterday & Tomorrow:

While visiting the Conservatoire National des Arts et Métiers museum in Paris, I captured the above photo of ROBOT HILARE 1, designed in 1977 at the LAAS. This groundbreaking mobile robot was among the first to navigate autonomously using ultrasound sensors to detect obstacles.

The evolution of ultrasonic technology, from simple obstacle detection in early robotics like the HILARE to today’s cross-device tracking with AI, showcases how machines are shifting from basic mechanical sensing to intelligent, adaptive systems. In the next stage, Generative AI combined with ultrasonic tracking is poised to bring a new dimension—real-time behavioral prediction, personalized surveillance, and automated decision-making like social engineering attempts based on behavioral profiles built from ultrasonic data, making attacks more convincing and also GenAI could autonomously decide when and how to deploy ultrasonic attacks, interact with or control devices, without human intervention.. This progression mirrors social learning, where machines will predict, react, and adjust their responses to human actions, functioning as proactive agents rather than passive tools.

This ability to gather data from multiple devices, analyze human behavior through ultrasonic signals, and adapt surveillance strategies in real-time signifies a transformation. Machines evolve from tracking movements to predicting thoughts and preferences.

Future Implications of Ultrasonic Technology + GenAI

Imagine in a wartime scenario, AI and ultrasonic technology are used to not just track movements, but also to manipulate how people think and act. Through real-time adaptation, AI can deliver misinformation—like news reports or social media posts—based on individuals’ behavior, preying on fears and insecurities. Cognitive influences and Subtle cues, such as subliminal messages or social comparisons (e.g., “Others are preparing for this, why aren’t you?”), can steer entire groups toward certain decisions without them realizing. Based on this people’s profile data, AI can deliver fear-based misinformation or subtle psychological cues—such as unsettling news reports or biased advertisements—through their devices at vulnerable moments. The ultrasonic tracking enables the AI to adjust its messaging based on real-time activity and location. This is a form of psychological war that happens invisibly, shaping the choices of citizens, or even entire nations.

Also, if people in displaced areas or stuck at home under stress or fear, and ultrasonic tracking is used to monitor their movements or proximity to devices like TVs or speakers. These devices pick up their location or activity, and AI then predicts their emotional state (Something to cover in future posts). This makes the psychological influence highly adaptive, targeting people when they are most likely to be influenced, such as when feeling isolated or uncertain.

This blending of surveillance with targeted, dynamic manipulation would allow for greater control over public sentiment, trust, and cohesion, with devastating effects on collective decision-making. For instance, misleading information tailored to specific ethnic or political groups could inflame tensions, disrupting unity in a way that is hard to counter once it spreads across digital platforms. The lines between tracking, surveillance, and influencing cognitive processes become blurred, making it easier to manipulate public opinion in real time.

This will likely lead to even more interconnected systems where your environment interacts with your devices in ways you can’t easily detect.

To conclude, ultrasonic tracking is just one method within a broader arsenal used to capture and analyze human behavior. It provides part of the picture, but not the full spectrum of human actions, thoughts, and motivations. In our upcoming series for paid newsletter subscribers, we will explore other technologies that contribute to a deeper understanding of behavior and discuss ways to defend against these invasive techniques.

This is part of my AI Security Newsletter. If you are interested in this topic and wants to receive future posts, you can subscribe below:

References

1. UNSW, “Ultrasonic Cold Brew Coffee Ready Under Three Minutes,” UNSW Newsroom, May 2024.
2. Zhe Yang, Yifei Wang, Yun Pan, “Inaudible Sounds From Appliances as Anchors: A New Signal of Opportunity for Indoor Localization,” IEEE Sensors Journal, December 2022.
3. Chen Yan, “EchoAttack: Practical Inaudible Attacks to Smart Earbuds,” Conference Paper, 2020.
4. Guoming Zhang et al., “DolphinAttack: Inaudible Voice Commands,” ACM, 2017.
5. Guanhong Tao et al., “SurfingAttack: Interactive Hidden Attack on Voice Assistants Using Ultrasonic Guided Waves,” Network and Distributed Systems Security Symposium (NDSS), 2020.
6. Lei Shi et al., “Ultrasound Communication Using the Nonlinearity Effect of Microphone Circuits in Smart Devices,” IEEE, 2019.
7. Image: https://svantek.com/academy/sound-wave/