Making your chatbot work with your humans
The key idea is this: if our support chatbot can’t answer every question – as indeed it never will – then there has to be a way for the chatbot to let a human being respond instead. If a human being does respond, then the chatbot should learn that that is how it should have responded.
If the chatbot can learn, then we don’t need to do any up-front programming at all, we can just let the chatbot learn from past conversations. Or even have the chatbot be completely naive when it is first turned on, so that it always passes back to a human agent until it has built up some knowledge.
Whatever approach you take, you will end up with a corpus of conversations: lots of examples of users asking something, getting a response from support, clarifying what they want, and then getting an answer.
Predicting the appropriate thing to say next becomes a machine learning problem: given a new, otherwise unseen data blob, predict which category it belongs to. The data blobs are all the things that have been said so
6
far in the dialog, and the category is whatever it is that a human support desk agent is most likely to have said as a response.
It looks like artificial intelligence. This is AI for ITSM.
Solving machine learning problems with natural language text
There is a rich mine of research articles and a lot of well-understood best practice about how to do machine learning problems with natural language text like this. Good solutions have been found in support vector machines, LTSM architectures for deep neural networks, word2vec embedding of sentences. If you have a team of data scientists on staff, you can ask them about this.
It turns out that techniques from the 1960s work well enough that you can code up a solution in a few hours. I used a bag-of-words model combined with logistic regression and I get quite acceptable results. (At this point, almost any serious data scientist or AI guru should rightly be snickering in the background, but bear with me.)
The bag-of-words model says that when a user asks something, you can ignore the structure and grammar of what they’ve written and just focus on key words. If a user mentions “password” you probably don’t even need to know the rest of the sentence: you know what sort of support call this is. If they mention “Windows” the likely next response is almost always “have you tried rebooting it yet?”
If you speak a language with 70,000
different words (in all their variations,