The past couple of years have ushered in an exciting age for Natural Language Processing using deep neural networks. Research in the field of using deep pre-trained models has resulted in a massive leap in state-of-the-art results for many of the NLP tasks, such as text classification, natural language inference and question-answering.
Some of the key milestones have been ELMo, ULMFiTand OpenAI Transformer. All these approaches allow us to pre-train an unsupervised language model on large corpus of data such as all wikipedia articles, and then fine-tune these pre-trained models on downstream tasks.
Perhaps the most exciting event in this area has been the release of BERT, a multilingual transformer based model that has achieved state-of-the-art results on various NLP tasks. BERT is a bidirectional model that is based on the transformer architecture, it replaces the sequential nature of RNN (LSTM & GRU) with a much faster attention-based approach. The model is also pre-trained on two unsupervised tasks, masked language modeling and next sentence prediction. This allows us to use a pre-trained BERT model by fine-tuning the same on downstream specific tasks such as sentiment or emotion classification, intent detection, question answering and more.
At Utterworks we have recognised the game-changing leap in the capabilities of Natural Language Processing models to solve complex enterprise and social challenges (particularly in customer service and information accessibility) where unstructured text and natural language are an untapped resource. Check out the solutions we have developed in this area for more information.
Fluent Find
Make finding Help & Support a great customer experience. We’ve written about the challenges with website search for Help & Support. In short, keyword matching isn’t up to the task of giving customers…
Fluent Converse
Have great conversations with customers. We believe that with Conversational AI you can simplify your interactions with customers, reduce process time, improve data quality, and increase customer and advisor satisfaction. It’s no secret…
Natural Language Services
The evolution of Natural Language Processing into a powerful enterprise capability has opened up a new seam of value from information: extracting insight from unstructured text. Many organisations have plentiful reserves of unstructured…