The first chatbot created in 1966 by Joseph Weizenbaum wasn’t too smart. In fact, it wasn’t event called chatbot, this term was coined much later. Nevertheless, it was doing exactly what modern chatbots are doing: answering questions asked by the user and even asking questions by itself. Weizenbaum named it Eliza after Eliza Doolittle from Bernard Shaw’s “Pygmalion”.
Eliza followed a pattern-based script. On a basic level, a script was a set of rules that defined what the chatbot should answer if certain keywords were found in the text typed by the user. For example, a if the user asked “Are you a robot?” or “I’m wondering if you are a robot”, Eliza should answer “Yes, I am” relying on keywords, like “robot” in conjunction with “you”.
Eliza also used the substitution methodology when the first person and second person pronouns were switched. For example, if the user said “I am upset”, Eliza could answer “I’m sorry to hear that you are upset”.
A combination of these techniques made some people (including the Weizenbaum’s own secretary) think that they were chatting with a real person. To some extent, their conversations did look like a dialog with a real person. One of the most famous examples shows a dialog between a human and Eliza that followed a script called Doctor:
Human: Men are all alike.
ELIZA: In what way?
Human: They’re always bugging us about something or other.
ELIZA: Can you think of a specific example?
Human: Well, my boyfriend made me come here.
ELIZA: Your boyfriend made you come here?
Human: He says I’m depressed much of the time.
ELIZA: I am sorry to hear that you are depressed.
Human: It’s true. I am unhappy.
ELIZA: Do you think coming here will help you not to be unhappy?
Human: I need some help, that much seems certain.
ELIZA: What would it mean to you if you got some help?
Human: Perhaps I could learn to get along with my mother.
ELIZA: Tell me more about your family.
Although for a moment you could think that you are talking to another human, any deviation from the pre-defined conversation flow would immediately make Eliza to fail or begin responding in an akward way.
Fast forward to 2017, most of chatbots are still using the same pattern-based approach. In our days, it’s much more advanced and allows for far more flexibility, but chatbot’s answers are still mainly determined by the pre-defined patterns.
It’s not necessarily bad, though. In some cases, a pattern-based chatbot can be a perfect and affordable answer to your needs. But in some situations, you would need a smarter companion. In this post, I’ll show you two major approaches to building chatbots. We’ll also see how semantically structured content can let chatbots go far beyond the boundaries of a pattern-driven conversation.
Let’s suppose that we are building a chatbot that gives a recipe of a requested dish. The recipe needs to be adjusted based on various parameters, such as user’s culinary skills, willingness to add some extras, expected cooking time, and so on.
A simple conversation flow may go like this:
Chatbot: Welcome! I’m a recipe chatbot. What would you like to cook?
Chatbot: Excellent choice! I can recommend adding ham, tomatoes, and cheese to your omelet. Would you like to add any of these?
User: Ham and cheese
Chatbot: To let me find the recipe that matches your culinary skills, please tell me if you are a culinary pro, intermediate, or beginner?
User: I’m pro
Chatbot: Here’s your recipe…
Provided that we have recipes semantically structured and provided with semantic metadata, implementing such a chatbot will be relatively straightforward.
If you are not familiar with how chatbots work and recognize user’s input, you may want to read a great summary of my webinar on chatbots written by Marcia Riefer Johnston of The Content Wrangler.
But here comes a challenge. If we want to build a chatbot that creates a really great user experience, providing a piece of content as a response to the direct user’s question is just one part of the task. What differentiate a good user experience from a great one is two things:
This is when an ontology-based chatbot empowered by natural language processing (NLP) capabilities comes to the stage.
An ontology represents a structural representation of the domain’s entities and relationships between them. For example, our culinary domain in a very shortened and simplified way could look something like this:
It may look like a regular metadata model, but ontology is more than because it also defines relationships between the entities. For example, we may define a relationship of type “X goes well with Y”:
And this is where magic happens. Now, when we know the relationships between the entities, the questions that the chatbot asks can be generated dynamically based on these relationships.
Suppose that the user asks: “What can I have for breakfast?” Now the chatbot can do the following:
The chatbot has gathered all required information by asking questions that were dynamically defined by the ontology (as opposed to explicitly defining all possible intents and entities in case of a pattern-based chatbot) and can generate a JSON representation of the user’s context:
Now, provided that our recipes have semantic metadata and markup, the chatbot needs to match the JSON to the metadata to find the relevant content, which I’ve already discussed in my previous publications, like this article in the latest issue of Communicator magazine.
Pattern-based chatbots are good in finding relevant information when there is only one piece of information that answers the user’s question and there is a small number of intents and utterances for each intent. Adding new intents, utterances, and dependencies between intents as the amount and complexity of your content grows overtime will quickly become an overhead. An alternative approach that you may want to consider is to set up an ontology that will be used as a way to navigate through the options available to the user and dynamically generate questions to gather information about the user’s context. Although this approach seems to be more expensive in the short run, in the long run, it turns out to be more flexible and beneficial for both end user (content consumer) and chatbot developer.