What is Natural Language Processing?
Natural Language Processing, or NLP, is the process of extracting the meaning, or intent, behind human language. In the field of Conversational artificial intelligence (AI), NLP allows machines and applications to understand the intent of human language inputs, and then generate appropriate responses, resulting in a natural conversation flow.
Most commonly, NLP is used as an umbrella term to include Natural Language Understanding (NLU), Natural Language Generation (NLG), and Dialog Management.
What is the importance of intent?
Human language is extremely complex. Depending on the speaker, situation and cultural bias, words can mean different things in different contexts. NLP enables machines and software applications to make sense of a human language, recognize intent despite the order of words or the way they are used, and produce an appropriate response.
What are some use cases of natural language processing?
Examples of NLP are all around us. Smart Speakers can tell you the weather and set a timer, cars can respond to voice commands, and virtual assistants can help you accomplish customer service tasks without engaging an agent. However, we as humans, being the experts of human language, can easily spot good NLP from a clunky one.
What are some examples of bad NLP?
If the NLP behind an application is poorly designed or not advanced enough, you will find yourself having to change the structure of your phrase to dictate the appropriate response. For example, if you are booking a hotel reservation, and say “I am booking a room for my wife and I”, the machine may not recognize that wife and I implies two people. Instead, in the example of this poor NLP, you would have to be more specific and say “I am booking a room for two people.”
What are some examples of good NLP?
Advanced NLP is virtually indistinguishable from speaking with a human. This means that if you say “My order was shipped to the wrong address, I would like to get a refund,” the system understands that you need to cancel an order, rather than proceed with a shipping issue. Without recognizing the true intent, this may have caused multiple transfers and repetition, and a frustrating experience for the customer.
How does NLP work?
Let’s take a deeper dive into the technical aspects of NLP:
Natural Language Understanding (NLU): Natural language understanding or natural-language interpretation is a subset of natural language processing in artificial intelligence that covers machine reading comprehension. NLU helps machines understand spoken or written human language.
Natural Language Generation (NLG): Natural-language generation is another subset of NLP that converts structured data into natural language. In other words, NLG is the process of producing words, phrases and sentences that have contextual meaning and could be understood by humans.
Dialog Management (DM): Dialog management is one of the most important parts of NLP. Dialog Management determines the actual context of the dialogue and offers state and flow of the dialog to make the conversation human-like. In simpler terms, the dialog management keeps the conversation flowing with appropriate responses and questions.
The definition of NLP could also be stretched to include sentiment analysis, information (as in entity, intent, relationship) extraction and information retrieval.
Sentiment Analysis: Sentiment analysis is the process of using natural language processing and other branches of AI such as text analysis, biometrics etc. to identify, extract, study affective states of human emotion and subjective information. For example, if a user begins the call saying “Representative,” there could be a different underlying intent whether the tone is angry or calm.
Natural Language Processing Use
The usage of NLP is highly contextual and driven by intention. We, at Interactions, use Natural Language Processing in customer service transactions, to extract the meaning with an intention of having a conversation with the person. Other applications of AI such as search engines, use NLP with an intention of information or document retrieval. Machine Translation systems also extract meaning, with the intention of moving the meaning over to the target language, ex from english to french or vice versa.
Meaning can be comprised and influenced by multiple different dimensions, such as:
- What the user wants to do
- The user’s emotions
- The user’s demographics.
- Past context of a conversation or transaction
NLP extracts the meaning, using the above influences and more, with an intention of having a conversation with the person at a human level.
Variables that can impact intent:
NLP has come a long way. But, as the human language evolves to include more variables, the implied intent of spoken words becomes more difficult. This is especially true in a customer service setting, where there can be a diverse customer base calling.
Some of these variables are:
Languages and dialects
A simple example of this can be seen in the difference of British and American English, where different phrases and words can have different intentions.
Slang
New words and phrases are constantly being created. When virtual assistants hear them for the first time, they may have trouble extracting the meaning behind the words.
Tone
The same sentence can be interpreted many ways depending on the customers tone. Even a phrase as simple as “Great, thanks” with a sarcastic tone can have a completely different implementation. It is important for NLP to be able to comprehend the tone in order to best respond.
Background noise (or lack thereof)
Another variable in determining intent is whether or not there is background noise on the call, which helps establish context. For example, if an NLP system identifies that a customer is calling their insurance company from a noisy, outdoor place, like at the scene of a car crash, versus a quiet indoor location, the system could deliver more appropriate dialog flow for a better experience.
The Future of NLP
Just as humans become better at communicating as they mature, NLP will continue to advance and offer more functionally and benefits to speech technology. With increased focus put on data-driven interactions, Conversational AI technology will leverage NLP for conversations that are more personalized, accurate, and natural.
What will this look like? Think about a search engine. You type in a series of words and hope that the search engine will know what you want to find. But you may have to try a few different combinations of words and phrasing to get it right. As NLP advances, the search will become more and more accurate. In other words, the machine can better understand your intent on the first try. This occurs through more advanced modeling of the AI and larger pools of data to drive results.
Customer experience applications will work the same way. The AI technology will become more efficient at understanding exactly what the customer is needing, whether via text or voice channels. This will lead to a more natural conversation and less reliance on human agents.
Do you ever ask for a representative when you get on the phone with a brand because you know you need a human to understand your problem? As NLP advances, this occurrence will decrease. Customers will be able to get more done with self-service technology and frustration with automated systems will be eliminated.
