Laytoun' thoughts!

Getting started with Built-in NLP for Messenger Bots

Mohammed Aboullaite -

Messenger team recently announced a bunch of exciting new features and tools to enrich the bot experience. But there is one cool feature that we will talk about in this post: The built-in NLP feature and Customizing NLP using Wit.ai. You can checkout the code on Github. Also, I'm a java fan, So the code is written in java

Our Bot simply tries to describe the weather of whatever City you mention:

If you've not built a chat bot before, please read my other tutorial first.

Using Messenger Built-in NLP Only.

Before start using the built-in NLP on Messenger you should first enable it. Messenger documentation contains a great and clean explanation for this.

By default, Messenger's built-in NLP detects the following entities in English only:

  • Greetings
  • Thanks
  • Bye

It also detects information like date, time, location, amount of money, phone number and email. For example, if someone sends the message, “Hii” you will get something like this from the webhook:

        {  
           "object":"page",
           "entry":[  
              {  
                 "id":"255502108230287",
                 "time":1502382558816,
                 "messaging":[  
                    {  
                       "sender":{  
                          "id":"1573650332660059"
                       },
                       "recipient":{  
                          "id":"255502108230287"
                       },
                       "timestamp":1502381982109,
                       "message":{  
                          "mid":"mid.$cAACylCZ7gBtj_jcJnVdzPHBbHOTC",
                          "seq":302583,
                          "text":"hii",
                          "nlp":{  
                             "entities":{  
                                "location":[  
                                   {  
                                      "suggested":true,
                                      "confidence":0.97574394420685,
                                      "value":"hii",
                                      "type":"value"
                                   }
                                ],
                                "greetings":[  
                                   {  
                                      "confidence":0.71404528798453,
                                      "value":"true"
                                   }
                                ]
                             }
                          }
                       }
                    }
                 ]
              }
           ]
        }

For each message, the Messenger Platform will return a mapping of the entities that were captured alongside their structured data. The key pieces of information here are the confidence and the value for each entity.

  • confidence represents the confidence level of the next step, between 0 (low) and 1 (high).
  • value is the parser output.

Using Wit.ai

  1. Sign up for a Wit.ai account.
  2. Find the app token in your weather app so we can test everything is working. In your app go to the Settings page then find the “Server Access Token” in API Details.
  3. Now, go to your app's 'Messenger Settings' page.
  4. In the 'Built-in NLP' section, select one of your subscribed Pages.
  5. Add your Wit Server access token. You can find your access token in the Wit App settings.
  6. Save your settings.

That's all, you've successfully connected Messenger and Wit.ai for a more rich NLP experience.

Now we should tell Wit.ai what to do with users inputs, and what we're exactly looking for. The gif below briefly explains how to do this:

We simply say to Wit to detect if the user mention any location in his sentence, and extract.

Show me the code

The app expose 3 endpoints:

  • GET /ping: Health check endpoint

  • GET /webhook: Callback URL Verification

  • POST /webhook: To receive all events from the webhook

      @RequestMapping(value = "ping", method = RequestMethod.GET)
      public String health () {
          return "pong";
      }
      @RequestMapping(value = "webhook", method = RequestMethod.GET)
      public String Checkwebhook (@RequestParam("hub.verify_token") final String verify_token,
                                 @RequestParam("hub.challenge") final String challenge) {
          if (this.validationToken.equals(verify_token) && challenge != null) {
              return challenge;
          }
          return "Error, wrong validation token";
      }
      @RequestMapping(value = "webhook", method = RequestMethod.POST)
      public void HandleWebhook (@RequestBody final MessageReceived messages , HttpServletRequest request) throws IOException{
          Optional<List<MessageReceived.Messaging>> messagingList = messages.entry.stream().map(e -> e.messaging).filter(message -> message.stream().anyMatch(m -> m.message != null && m.message.text !=null)).findFirst();
          if(messagingList.isPresent())
          messagingList.get().forEach(m -> {
              final MessageResponse response = new MessageResponse();
              response.recipient = new MessageResponse.Recipient();
              response.recipient.id = m.sender.id;
              response.message = new MessageResponse.MessageData();
              response.message.text = messageHandler.handleMessage(m.message.nlp.entities);
              this.send.sendMessage(response);
          });
      }

After receiving a message from the webhook, The method MessageHandler.handleMessage checks if it's a Greeting or a Location message (using entities). We filter also results based on confidence using a threshold (0.8 in macase). We return a generic message if nothing applies. I should also mention that I extract weather information from Openweathermap.

public  String handleMessage(Map<String,List<Map<String,Object>>> entities){
    if(EntityMap.entityExist(GREETINGS_MSG, entities))
     return  EntityMap.confidence(GREETINGS_MSG, entities) > TRESHOLD ? "Hello Sir!" : GENERIC_MSG;
    if(EntityMap.entityExist(LOCATION_MSG, entities))
        return  EntityMap.confidence(LOCATION_MSG, entities) > TRESHOLD ?  Weather.weather(EntityMap.value(LOCATION_MSG, entities), omwtoken) : GENERIC_MSG;

    return  GENERIC_MSG;

}

Deploy the app

You need to set up first 3 env variables:

You'll need also a free hosting service like Heroko or Clever Cloud, or you can simply use NGROK to set up your webhook!

That's all ;) If you've any comment or suggestion Please feel free to comment below.