For now, voice assistant speakers from Google and Amazon only start recording after you press a button or say a startup command like “Hey, Alexa” or “OK, Google.” It’s a comforting restriction at a time when the security of our personal information is at a premium. But that peace of mind may be a thing of the past if patents from major smart speaker brands ever see the light of day.
Amazon filed a patent for an algorithm that would allow its Alexa assistant to listen in on any conversation without requiring a wake word. Called “voice sniffer,” the algorithm could be implanted into a range of mobile devices (eBooks, tablets, smartphones) to always listen for key “trigger” words like “prefer,” “like,” “love,” or “downloaded” and combine those with “adjacent audio that can be analyzed.” Any verb indicating some level of interest in an activity or product could be used to generate a profile of the user for advertising purposes. Words used more often indicate a greater level of interest and would, therefore, gain priority.
An example offered in the pending patent describes how the voice analysis could determine the type of interest in a particular product or activity.
“A phrase such as “love to paint” might result in a keyword to be associated with a user, but a phrase such as “hate to draw” might also result in a keyword to be associated with that user,” the patent says. “Since each trigger word indicates a different type of interest, an algorithm might also generate a flag, identifier, or other indicia for at least one of the keywords to indicate whether there is positive or negative interest in that keyword.”
The collected recordings would be sent to a content provider or advertiser where the keywords are stored and used to generate content that is relevant to the user. It’s similar to ad-targeting tactics used today but with a heavy reliance on recording your conversations. A diagram included in the patent explains how Alexa could even listen to a phone call and provide recommendations after the user hangs up. In one example, a user is provided restaurant suggestions and the ability to reserve a table after mentioning they would like to eat out.
The most concerning method described in the patent is the use of video cameras. Amazon says “one or more video cameras” would be used to capture information to figure out who is speaking based on lip movements or other indicators.
The benefits of this intrusive technique are clear for a company like Amazon. The largest online retailer in the world could use voice clippings to encourage users to purchase more items or give them exclusive offers.
But not every Echo customer will warm up to the idea that their every conversation is being listened to, especially given the recent privacy scandals plaguing other tech giants. Fortunately, they won’t have to. The patent, filed in 2017, says users have the option to activate or deactivate the “sniffing or voice capture processes” for privacy and security purposes. It’s not clear if there would be an opt-in page presented before the voice sniffer is activated, but Amazon says a user must show a “willingness to have voice content analyzed.”
In response to concerns regarding the patent, an Amazon spokesperson told ABC News that it’s serious about user privacy.
“We take privacy seriously and have built multiple layers of privacy into our devices. We do not use customers’ voice recordings for targeted advertising. Like many companies, we file a number of forward-looking patent applications that explore the full possibilities of new technology. Patents take multiple years to receive and do not necessarily reflect current developments to products and services.”
The patent has not yet been approved by the United States Patent and Trademark Office (USPTO), and even if it is, there is no guarantee that the voice recognition technology will ever make its way into an Echo speaker. Mega-corporations like Amazon file hundreds, if not thousands, of patents each year.
A similar patent was filed by its smart speaker rival Google, which owns a significant portion of the market with its Home speakers. In its version, the smart speaker could determine whether a child is engaging in “mischief” at home by analyzing their location and speech. Other Google patents describe using the observations of a Nest video camera to serve up ads, listening to conversations without a wake word, and creating personalized profiles for each smart speaker user.
Google released a statement after Santa Monica, California-based nonprofit Consumer Watchdog accused it of designing “spyware and a surveillance system meant to serve you up to advertisers.”
“All devices that come with the Google Assistant, including Google Home, are designed with user privacy in mind,” Google told the New York Times. “For Google Home, we only store voice queries after a physical trigger or after recognizing a hotword trigger like, ‘OK, Google’ or ‘Hey, Google. We file patent applications on a variety of ideas that our employees come up with. Some of those ideas later mature into real products or services, some don’t. Prospective product announcements should not necessarily be inferred from our patent applications.”
Smart speakers are one of the hottest tech categories of the past few years. It was recently determined that one in five adults, or around 47.3 million people, have access to these devices.
