As streaming shatters, here’s how Amazon wants to glue it back together with AI

Good news, everyone! Amazon already has tipped its hand as to what’s in store for Fire TV in 2024. At least, some of what’s in store.

Flashback to September 2023 at its fall products announcements at HQ2 in Arlington, Virginia, when Amazon showed off new hardware with new features — specifically, the refreshed Fire TV Stick Max 4K and along with it, the latest Ambient Experience. Good stuff, all around.

That’s on the hardware side of things. Software is always improving, and Amazon said these major features are coming in the first quarter of 2024:

• Updated search results that leverage the power of large language models. (Call it AI search, if you must.)
• The ability to create AI-manipulated backgrounds from your own photos.

Two of those things directly relate to the major leaps in AI processing that Amazon (and others) put front and center in 2023. And in this case, we, as end users, will get to leverage them in early 2024. And that’s just the stuff we know about. (A third new feature, the ability for apps to be included in the “Continue watching” row of the home screen, is already available, but look for more apps to be included in the future.)

We chatted about all that with Joshua Danovitz, Director of Fire TV Experience at Amazon. And then some.

Better search through AI

If it seems like every tech company in the world cares more about large language models than anything else these days, you’re not necessarily wrong. Or, at the very least, LLMs have quickly gained a more prominent place in public-facing features. And that’s going to be extremely apparent in the search results you get when using Amazon Fire TV. We got a taste of that at Amazon’s September event, and we’ll get to try it ourselves early in 2024.

“We’re already are very good at helping customers when they know what they want to watch,” Danovitz said before throwing out some examples. “Find me Tom Cruise movies. Show me comedies or rom-com movies. Or if you know the name of the title. Easy. We’re very good at that.”

That’s a basic search — table stakes for a company like Amazon in 2023. The edge cases — where you don’t necessarily know what you’re looking for — are more tricky. And that’s what all this “AI search” stuff is trying to address. And it’s doing so in a way that’s less mechanical and more like how you’d ask a human being.

“What we’re doing with LLM,” Danovitz continued, “is making it so that you’re able to have an interaction and then a multi-turn, almost conversational style, when you don’t really know (what it is you’re looking for).”

Those of us of a certain age can recall what it was like to go into a video rental store and run a real-time search. With a person.

“If you if you rewind 20 years, you go into your local video store and say things like, ‘Hey, what was the movie where somebody was, you know, stuck in a submarine?’ You know where you’re trying to go, but you don’t actually remember all the details. That’s the type of thing that the LLM search will help with.”

Refining results is another major use case of LLM-powered search, Danovitz said.

“The next thing will happen is when you don’t know exactly what the title is, but you’re like, ‘Hey, I would like to find cartoon movies with dinosaurs. But just the free ones.’ Today, that’s not something that we’re able to provide.”

It’s all about simplifying things for those of us who are just trying to watch something, especially in a space as fragmented — and as ever-changing — as streaming.

“Discovery of video is a journey, and it’s hard today,” Danovitz said. “Like, to be honest, I think that one of the trends that is often talked about is consolidation of content, how it’s getting moved around. It’s becoming harder for customers. We used to think it was getting harder with, you know, hundreds of cable channels or the first few streaming apps. It’s actually getting more complicated with all of the different bundles, name changes, pricing changes move to different tiers.”

Expand the Ambient Experience

The Amazon Fire TV Stick 4K Max is the first device outside of the Omni QLED line of Fire TVs to include the Ambient Experience. In 2024, we hope to see Ambient Experience come to additional Fire TV devices because it’s just that good.

Said Danovitz: “This was the first time that we decided, like, ‘Hey, let’s not just do it on televisions with sensors, let’s do it on a device that doesn’t have a sensor — a stick — because it’s a very different way that customers interact.’ ”

Of course, this isn’t quite as simple as it sounds, and there’s a reason that Ambient Experience wasn’t initially rolled out across the greater Fire TV lineup. And that reason wasn’t just to sell more 4K Max sticks.

“Some of ambient is more processor-intensive,” Danovitz said. “Some of it uses more AI, some of it uses connection to other services and back end. And so it definitely can be per hardware device.”

Software is always at the mercy of the hardware on which it runs. And Danovitz wouldn’t say if we could expect Ambient Experience on more existing hardware — you’d have to believe the Fire TV Cube probably is powerful enough — or if we’ll have to wait for new hardware updates that would require users to purchase anew. But we can, at least, expect the current implementation of Ambient Experience to improve.

“We’re getting a lot of very good feedback from customers directly,” Danovitz said. “But also, like, even on our home page where people are interacting with it. So it’s really exciting, and we’re continuing to invest in additional ambient features and [user experience] to actually get to the feature as well.”

The Omni QLED televisions have onboard sensors to detect if someone is in the room and within sight of the TV before triggering the Ambient Experience features. Fire TV sticks don’t have that. But they can talk to other devices via APIs, like, say, from something in the Amazon Echo family, the Ring doorbell, or possibly even the Eero family of wireless routers. And that could potentially give the Fire TV Ambient Experience more data as to whether someone is in the room or not.

“There’s actually lots of interesting things that you can do with other sensors in your house,” Danovitz said. “We actually have lots of interactions from customers that could be helpful for television interaction as well. … Are there other ways to understand when there’s someone there or when you should trigger something? We’re exploring some of that.”

AI art is coming

Another fun feature we’re waiting on is AI art. The demo Amazon’s Daniel Rausch showed at the September event seemed simple enough and not unlike what any number of phone apps have done — take your images, run them through some AI processing, and output something new. Something more artsy.

Right now, Danovitz said, the focus is on some of the core features. “Like, I’m having a dinner party, and I want there to be artwork or something cute that I’m going to create on my own.” That’s “the most important thing for us to demonstrate to customers.”

Of course, that’s fraught with danger because people are people and will test the limits of what Amazon is willing to create with user-generated content. We’ll have to see how far out those boundaries lie. But for folks just looking to create something new — without trying to cause any scandal in the process — it’s going to be a cool feature.

New hardware in 2024

Something Danovitz definitely (and understandably) wouldn’t get into was any new Fire TV hardware coming in 2024. But given the cadence of product releases, it really shouldn’t surprise anyone to see something new coming down the pike from the Fire TV team. The question is what and when.

Only the top end of the Fire TV Stick lineup was refreshed in September 2023 — the Fire TV Stick 4K Max and the non-Max version. These impending updates helped lift the Fire TV Stick 4K Max to become the most innovative streaming device released this past year.

Looking further down the line, the Fire TV Stick and Fire TV Stick Lite were last refreshed in 2020. They’re likely due for updates, which would refresh the bottom end of the line. The Fire TV Cube is a tougher call. It got new hardware in late 2022, but it’s hard to say that it sits atop the lineup at this point, given all that’s in the Fire TV Stick 4K Max now, including Ambient Experience, which the Cube lacks.

That doesn’t necessarily mean you’ll have to buy new hardware in 2024 if you want to get the new features. But as mentioned before, a good bit of it is hardware-dependent. So, if you’ve been holding off on getting new Fire TV hardware, 2024 may be a good year to do so.

“We appreciate that customers like to — like, they bought something a year or two ago and would like to keep it,” Danovitz said. “We keep adding features as much as we can to all of those devices because it’s good when it gets smarter.”