Smart TVs are evidently so smart they can tell what's playing on the screen and show you pop-up ads based on what you watch. So discovered security researcher Paul McMillan this week while watching Inglorious Basterds on his Samsung smart TV: one minute into the movie, an Army recruitment ad popped up on the screen.
This is my Samsung SmartTV with a US Army pop-up ad on top of Inglorious Basterds streamed from Netflix on my FireTV pic.twitter.com/wuONS4VUfg
— Paul McMillan (@PaulM) January 19, 2016
Now, pop-up ads on Samsung and other smart TVs have been discovered before. But the weird thing here is that the TV can seemingly recognize any input you play through it, and add ads on top. What's more, the ads may be targeted based on content recognition, a sort of built-in Shazam for ads.
McMillan was watching the movie through an Amazon Fire set top box, and as an experiment, tried playing it from his computer connected to the TV through an HDMI cable. In both cases the Army ad appeared at the one-minute mark, leading McMillan to deduce that the ad was being served by Samsung, and that the internet-connected TV was using content recognition to show ads on top of any video coming in through the TV's input.
This seems to be a brand new kind of targeted advertising, McMillan told me. "In this case, it seems to be running some kind of watermarking or audio recognition system on top of anything that's playing," he said.
There is plenty of precedent for this practice.
"It's not only possible that SmartTVs are collecting viewing habit data for targeted advertising, it's already happening," Claire Gartland, consumer protection counsel at the Electronic Privacy Information Center, told me. "And it's not just Samsung. LG and Vizio have also gotten attention lately for tracking viewers."
Smart TVs have embedded "automatic content recognition" technology that's analyzing viewing habits and "sending data to third parties on everything you watch"
Like most internet-connected devices, smart TVs can harvest and share a startling amount of information on you based on your user activity. But what's extra eerie is that, as a 2015 Consumer Reports report explains, the major smart TV brands have embedded "automatic content recognition" technology that's analyzing viewing habits and "sending data to third parties on everything you watch, whether it's a TV broadcast, a streaming movie, a YouTube video, or a DVD from your private collection."
The tech monitors the audio or video (or both) that's on the screen to create a "fingerprint" of the content that's used to determine what programming is being watched, the report explains.
That data is incredibly valuable for ratings companies and publishers, as has obvious potential to be used to run targeted ads on the screen. Consumers usually don't know they're unwittingly opting in to these always-on tracking features when agreeing to vague and overreaching end user terms.
And the privacy policies of Samsung, LG, and Vizio leave the door wide open for future content-based targeted ads.
Around the same time users noticed that Samsung was inserting pop-up Pepsi ads directly into third party streaming apps. The company eventually admitted the pop-ups were part of an ad partnership with Yahoo that was "supposed to be" opt-in, but wound up serving ads without permission, forcing users to go through labyrinthe menu diving to figure out how to disable the feature.
At that time, Samsung said it had no plans to serve ads in "the immediate future."
In other words, it's not unlikely that smart TV users will eventually start seeing ads pop up for airlines while watching a travel documentary, or ads for dating sites appear while you're streaming the latest rom-com.
I wonder how long it will take for Smart TVs to track people that come through your living room.
— Z Unger (@Zlatty) January 20, 2016
This is concerning for a few reasons. For one, being bombarded with ads in exchange for browsing the web free-of-charge is annoying enough, but when pop-ups show up on a TV you paid $1,800 for, on top of a set-top box or streaming service you also pay for, people rightfully start to get pissed.
What's more, smart TVs, and Samsung in particular, already have a reputation for having poor user experience, enough so that many people simply ignore the baked-in "smart" features that come with most high-def TVs. In this latest case, for instance, McMillan says he only connected the TV to the internet because an obnoxious notification kept popping up reminding him to connect—"I didn't want to use the smart part anyway," he said. Then as soon as he did, viola, an ad showed up.
To get the ad to go away, McMillan went into the "smart" section of the TV's settings and disagreed with all the privacy policies. Alternatively, you could block the TV from accessing the internet and factory reset it to avoid the constant notifications reminding you to connect to the web (which only appear if you've previously connected to the network).
McMillan is convinced the Army ad he saw was related to the movie it appeared next to. "[Samsung] is pretty clearly doing content identification; the ad network clearly knows what everyone's watching," he said. "And I assume—since the Terms of Service clearly don't prohibit it—I assume that they're selling that data to anyone wants to buy it."
It raises the question: what exactly are these ads paying for? Shouldn't paying for the TV plus the content streaming service subscription spare us from pop-ups?
You'd think so, but as a GigaOM article pointed out last year, everyone wants a piece of the lucrative streaming video space. The profit margins on consumer electronics is significantly lower than streaming services, where the real growth is.
And aside from being an additional revenue stream, the money from ads (especially targeted ads) may allow manufacturers to sell the TV at a lower price in order to beat out the competition, security expert Zlatko Unger told me. Consumers don't know the ad capabilities are built into the software; they just see the price tag on the TV.
It makes you wonder if we'll start seeing "freemium" TVs that follow the familiar ad-supported business model: pay less for a TV loaded up with ads. "Right now you can opt out of it, but I would not be surprised to find someone selling a subsidized TV that does not allow you to opt out at some point," said McMillan.
How far will it go? Are we going to need an Adblocker for our internet-connected TVs? In fact, many users have asked that question on various internet forums; AdBlock's support forum said they had no plans for a blocker for smart TVs.
"I expect to be commoditized if I'm reading free web pages on the internet," McMillan said. "I'm a little less happy about it when I paid a bunch of money for a product."
UPDATE: A Samsung spokesperson responded to my request for comment with this statement: "In the TV setup mode, Samsung smart TVs allow consumers to choose whether or not to receive customized advertising within the Smart Hub navigation interface. We encourage consumers to contact us directly at 1-800-SAMSUNG with product questions."