Facebook has unveiled a new in-house artificial intelligence engine called “DeepText”, built to understand the meaning and sentiment behind text on the social network.
Facebook said DeepText can understand with “near-human accuracy” the textual content of thousands posts per second in more than 20 languages. This ability is hugely important to Facebook, where much of the text users post is unstructured and hasn’t been categorized or analyzed based on intent or meaning.
The technology, which Facebook announced on Wednesday, has already been used to improve the ability of Messenger chat bots to connect users with cabs, detecting when users make a comment that indicates they need a ride. DeepText can differentiate between, for instance, someone saying, “I just came out of the taxi,” and “I need a ride.” The engine is also being applied to sales-related text. When a user posts about selling something, DeepText can extract information such as the type of item, its location and the cost, as well as guide the user as to how to promote the sale and where to post.
“If you want to have bots that communicate in a natural and intelligent way, text understanding is a critical piece of technology” said Hussein Mehanna, Facebook’s director of core machine learning. “People use text as a major form of communication on Facebook. There are thousands of posts per second, tens of millions of comments every day. With text understanding we can get closer and closer to human accuracy or near-human understanding.”
AI technology like DeepText will be key to Facebook’s efforts to compete with Google as a search platform and could dramatically advance other conversational tools in Messenger as well as the curation of content, such as comments, posts and ads, or the flagging of that content as spam. Celebrities and public figures, for example, can received thousands of comments on posts. Text understanding helps sort out which comments to highlight and which are inappropriate or unrelated.
Text understanding spans many roles, from interpreting what a post is about, to recognizing pieces of information like names or stats from a game. Facebook said it needed to use deep learning in order to understand subtleties of slang and the use of words that carry several meanings, such as “blackberry,” where natural language processing tools fell short. DeepText relies on neural networks, modelled after the human brain, in order to understand the relationships between characters, words and multiple languages. DeepText can build models that are language-agnostic, by using a process called “word embedding” to understand the relationship of words and phrases to one another instead of assigning individual words to an integer ID, as is traditionally done in natural language processing approaches. In DeepText, for example, closely associates “happy birthday” and “feliz cumpleaños” because the phrases have the same meaning even though they are in different languages.
DeepText was built on top of Facebook’s AI backbone, FbLearner, and uses Facebook pages, in part, as training data. Before DeepText, Facebook didn’t have a centralized text understanding engine, Mehanna said. The company used other technology, some of which were state of the art, but did not use deep learning. DeepText is overall about 20% more accurate than older generation technologies Facebook used before it, and should continue to improve in accuracy, Mehanna said. The technology also requires less human-labelled data than more traditional technologies.
Facebook is working to better marry visual content understanding with text understanding, for instance to interpret a friend’s photo of a baby with a caption like “Day 25.” “Deep learning is giving us significant amounts of accuracy above a lot of the traditional natural language processing techniques that exist already in academia,” Mehanna said. “Deep learning is really revolutionizing text understanding, just as it did with computer vision.”
The technology was built on ideas that were developed in papers by Facebook’s director of AI research Yann LeCun and research scientist Ronan Collobert. Facebook’s AI research group, FAIR, worked with applied machine learning teams to build the tool.
Feature image credit: https://fbnewsroomus.files.wordpress.com/2015/07/fbwm_cw_01.jpg?w=800