Voice search has traditionally always been a little bit hit-and-miss. Not only does your device have to accurately pick up your search query, but it also has to try to process that query to provide you with the most accurate results. Many services that offer voice search use a keyword-based search system that is effective for simple searches, but often falls down when given a more complex query.
This used to be the case with voice search on theFire TV, with a keyword-based search system that would bring up shows or movies if you searched by name but would struggle to return relevant results if your query was a little less clear.
In May, however, Amazon introduced a newAI-poweredsearch feature onFire TV devicesthat makes finding content far easier thanks to its ability to understand complex queries and generate results based on context rather than simply by keyword. The new Fire TV search is undoubtedly a significant improvement, but how does it work? I asked Jitesh Mehta, senior manager of software development at Amazon, to explain more.
6 Amazon Fire TV features that will change the way you watch
From content mirroring to controlling autoplay, it’s possible to make Amazon TV your entire media hub.
How the Fire TV’s AI-powered search works
An LLM allows you to search using natural language
Before the Fire TV introduced its AI-powered search capabilities, finding the right content could be difficult if you couldn’t remember the name of a show or movie, or the name of the actors that appeared in it. Searching for “the movie where Keanu Reeves plays a samurai” might have brought up a list ofKeanu Reeves movies, or a list of samurai movies, but it wouldn’t have homed in on the movie 47 Ronin. However, with the new Fire TV search, that’s exactly what you’re able to do.
It’s possible to include information in your searches such asgenres,topics,plot points,characters,actors, and evenlines of dialogfrom the show or movie.Amazon’s blog postback in May about the new feature included the following examples:
Using the new Fire TV search features makes finding the content that you want to watch much easier, even if you’re able to’t remember important details such as the name of the show or movie. It also makes it simpler to find new content to watch by allowing you to search not only by genre, plot point, or other categories, but also by a combination of multiple different criteria.
How Amazon developed the new Fire TV search
The company created a new semantic search system
The new Fire TV search capabilities are a big step forward from the previous search, making it far easier to find what you want to watch, but how was Amazon able to make the search tool so much more capable?According to Jitesh Mehta, there were three key aspects that needed to be developed.
“The model was customized for Fire TV search, and we trained it on Fire TV’s extensive entertainment content and app library, including data from IMDb. This gives Fire TV search the ability to reference plot points, character names, topics, notable quotes and scenes.”
The first is an entirely new semantic search system that not only considers the keywords within a search query, but also the context. For example, a search for “a movie about friendly aliens” that is focused purely on keywords might see the word “aliens” and return themovie of the same name, which is definitely not about the friendly kind. A context-based search understands the overall intent of the query is to find films in which the aliens are friendly and not trying to burst out of people’s stomachs. The results are therefore likely to be far more relevant.
This is where the power of large language models (LLMs) proves invaluable. An LLM is an artificial intelligence model that is trained on large amounts of data, making it able to interpret and generate natural language. Amazon’s own LLM was trained to understand the difference between certain types of queries, so that it could return the most relevant results based on the entire context of the query rather than simply the keywords.
The LLM used in Fire TV search was custom-made to power the feature. Mehta told me that “we built this experience from the ground up to help Fire TV customers more naturally and easily discover TV shows and movies that they want to watch. The model was customized for Fire TV search, and we trained it on Fire TV’s extensive entertainment content and app library, including data from IMDb. This gives Fire TV search the ability to reference plot points, character names, topics, notable quotes and scenes.”
Fire TV search builds on Alexa’s knowledge base
Alexa already has the information required for better search
Another key aspect in delivering the Fire TV’s new search capabilities was harnessingAlexa’s vast knowledge base. I tried asking Alexa on myEcho Show 5for the name of the film where Keanu Reeves plays a samurai, and she instantly gave me a snippet of information about the movie 47 Ronin.
“Utilizing the vast store of knowledge that already resides in Alexa means that Fire TV search is able to give much more accurate search results than were previously possible.”
Alexa already has the knowledge required to find the most relevant results for complex queries, but the previous version of Fire TV search didn’t have access to all of that information. Amazon has developed a way of allowing Fire TV search to leverage all the knowledge that Alexa can access. That information can then be used to help to bring up the most relevant search results.
Utilizing the vast store of knowledge that already resides in Alexa means that Fire TV search is able to give much more accurate search results than were previously possible. It means that you no longer need to Google “Keanu Reeves samurai movie” to find out what it’s called. Your Fire TV does all the heavy lifting for you, bringing you the results with far less effort.
Searching on Fire TV uses a three-pronged ranking system
The results also consider your own watch history and subscriptions
The team also needed to develop a more complex ranking system to determine which results would be the most likely to be relevant to any specific query. Where previously the search feature only needed to rank the results based on a keyword search, the new ranking system also has to incorporate results from the semantic search system, as well as information gleaned from Alexa’s knowledge database.
“The model takes several factors into account to personalize your results. It considers the semantic and lexical relevance match of your specific request, your personal watch history and preferences, IMDb ratings, what’s popular and trending with other customers, and your unique mix of subscriptions, so you are likely to see options that are included in your current subscriptions in the results.”
The new Fire TV search examines each specific query and then considers which information system is likely to offer the most useful results. For example, a search for “Ghostbusters” is likely to focus on the keyword ranking system to provide the most relevant results, whereas a search for “movies like Ghostbusters” will get the best results by using information from the new semantic search system. A query for “movies from the same year as Ghostbusters” would be best served by harnessing Alexa’s knowledge base to find information on the year that Ghostbusters was released.
According to Mehta, “the search system uses multiple information systems and uses our proprietary technology to rank and organize the search results.” By utilizing a combination of all three systems, Fire TV search attempts to produce the most accurate results for the type of query that it’s given.
The search query alone isn’t all that’s taken into account, however. Mehta explains that “the model takes several factors into account to personalize your results. It considers the semantic and lexical relevance match of your specific request, your personal watch history and preferences, IMDb ratings, what’s popular and trending with other customers, and your uniquemix of subscriptions, so you are likely to see options that are included in your current subscriptions in the results.” The overall outcome is that you’re presented with search results that are not only more accurate, but also personalized to you.
Fire TV search will continue to improve
There’s more to come for the Fire TV experience
The beauty of using AI to power your search is that you can continue to train it with real data from customer searches, to make it even better at dealing with complex search queries. According to Mehta, “as more customers use the new feature, Alexa and the Fire TV search system learns from the customer interactions to get smarter over time. For example, if a customer completes a complex search and clicks on one of the top results to start streaming, we take that as a positive signal. This feedback loop helps the AI behind the search system make regular tweaks to the search results, and customize it to customers' individual preferences.” All of which means that the more you use Fire TV search, the more useful it should become.
“This new Fire TV search feature is just the beginning of the next evolution of streaming, and we’re just getting started. We are planning to continue to evolve the feature over time, making it smarter, more personalized, and more intuitive.”
Amazon isn’t stopping there, however. Mehta told me that “this new Fire TV search feature is just the beginning of the next evolution of streaming, and we’re just getting started. We are planning to continue to evolve the feature over time, making it smarter, more personalized, and more intuitive.”
It’s not just Fire TV search that will continue to develop, either. “Fire TV search is only one part of the overall Fire TV Experience, which we’re always enhancing. Earlier this year, we launched a feature called AI Art, which combines generative AI and Alexa to enable customers to create artwork on their Fire TV screens using just their voice and their imaginations.” It will be interesting to see how the Fire TV will continue to harness new technology to improve the user experience.
Amazon Fire TV 55" Omni QLED Series 4K UHD smart TV
This 55-inch Amazon Fire TV has a stunning QLED display and the Fire TV Ambient experience that can turn your TV into a work of art.
