Analyze Real-time Sentiment, Emotion with IBM Watson

IBM Watson provides a broad suite of APIs to allow you to build in language, speech, vision, and insight capabilities into your app. We’ve already built a couple tutorials with IBM Watson, including a language translator chat app that translates messages between 11 languages in real time, as well as a text-to-speech app, that analyzes written text and speaks it out.

The underlying technology is PubNub BLOCKS, which lets you execute functions (like translating the text, or converting it to speech), on the data streams in motion. IBM Watson’s APIs are a perfect fit for this, and you can find a number of IBM Watson BLOCKS in the PubNub BLOCKS Catalog.

Analyzing User Sentiment and Context with IBM Watson

In this tutorial, we’ll introduce the IBM Watson BLOCK for Sentiment and Context Analysis, which allows you to gauge the user sentiment, emotion, tone, language, and concepts of an input. We’ll build an application that analyzes text inputs, and rates the user sentiment on a scale of 1-100 and ranking it as positive, negative, or neutral.

In the end, we’ll have something that looks like this:

Before we get into the tutorial, let’s check out the underlying IBM Watson AlchemyLanguage API.

IBM Watson AlchemyLanguage API

ibm watson sentiment analysis Automated sentiment analysis services are quite challenging to build and train on your own; they require substantial effort and engineering resources to maintain across a diverse array of application domains and user languages (not to mention immense compute resources and training sets!). In the meantime, the IBM Watson AlchemyLanguage API makes it easy to enable your applications with straightforward text sentiment analysis.

Looking closer at the API, sentiment analysis is just the beginning. There are a lot of API methods available for things like emotion analysis, entity detection, concepts, authors and more. It really is a powerful tool for distilling meaning from text. In this article though, we’ll keep it simple and just implement a basic thumbs-up, thumbs-down, neutral evaluation for user-generated fragments of text.

Getting Started with IBM Watson on Bluemix

You’ll need a PubNub account to get started. If you don’t have one, sign up here. If you do have one, head over to your Admin Dashboard and get your unique publish/subscribe keys (you’ll need those in a bit!).

Next, you’ll need an IBM Bluemix account to take advantage of the Watson APIs.

Step 1: Go to the Bluemix signup form.

Step 2: Go to the IBM Watson AlchemyLanguage API page and add it to your service dashboard.

Step 3: Go to the “service credentials” pane of the service instance you just created and make note of the “apikey” value.

Setting up the BLOCK

Step 1: Go to the application instance on the PubNub Admin Dashboard.

Step 2: Create a new BLOCK.

Step 3: Paste in the BLOCK code from the next section and update the credentials with the Watson credentials from the previous steps above.

Step 4: Start the BLOCK, and test it using the “publish message” button and payload on the left-hand side of the screen.

That’s all it takes to create your serverless code running in the cloud!

Diving into the Code – the BLOCK

You’ll want to grab the 27 lines of BLOCK JavaScript and save them to a file called pubnub_watson_sentiment.js. It’s available as a Gist on GitHub for your convenience.

First up, we declare our dependencies: xhr (for HTTP requests), and qs (for query string encoding).

const xhr = require('xhr');
const qs = require('codec/query_string');

Next, we create a method to handle incoming messages, declare the credential for accessing the AlchemyLanguage API and set up the URL for talking to the remote web service.

export default (request) => {
    const apiUrl = 'https://gateway-a.watsonplatform.net/calls/text/TextGetTextSentiment';
    const apiKey = '0000000000000000000000000000000000000000';

Next, we set up the IBM Watson web service query parameters. Note that we’re specifying JSON as the output format here since the sentiment analysis result is structured data.

  const querystring = qs.stringify({
      outputMode: 'json',
      showSourceText: false,
      text: request.message.text,
      apikey: apiKey
  });

Finally, we make a remote service request to the IBM Watson AlchemyLanguage API, and add message attributes with sentiment analysis of the given text. We return request.ok() to pass on the modified message object.

    return xhr.fetch(apiUrl + '?' + querystring)
        .then(function (r) {
            const body = JSON.parse(r.body);
            request.message.sentiment_score = body.docSentiment.score;
            request.message.sentiment_type  = body.docSentiment.type;
            return request.ok();
        })
        .catch(function (e){
            console.error(e);
            return request.ok();
        });
}

All in all, it doesn’t take a lot of code to add text sentiment analysis to your application. Next, the UI.

Diving into the Code – the User Interface

You’ll want to grab these 84 lines of HTML & JavaScript and save them to a file called pubnub_watson_sentiment.html.

The first thing you should do after saving the code is to replace two values in the JavaScript:

YOUR_PUB_KEY: with the PubNub publish key mentioned above.
YOUR_SUB_KEY: with the PubNub subscribe key mentioned above.

If you don’t, the UI will not be able to communicate with anything and probably clutter your console log with entirely too many errors.

For your convenience, this code is also available as a Gist on GitHub, and a Codepen as well. Enjoy!

Dependencies

First up, we have the JavaScript code & CSS dependencies of our application.

<!doctype html>
<html>
<head>
  <script src="https://cdn.pubnub.com/pubnub-3.15.1.min.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.6/angular.min.js"></script>
  <script src="https://cdn.pubnub.com/sdk/pubnub-angular/pubnub-angular-3.2.1.min.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>
  <link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.0.2/css/bootstrap.min.css" />
  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.3/css/font-awesome.min.css" />
</head>
<body>

For folks who have done front-end implementation with AngularJS before, these should be the usual suspects:

PubNub JavaScript client: to connect to our data stream integration channel.
AngularJS: were you expecting a niftier front-end framework? Impossible!
PubNub Angular JavaScript client: provides PubNub services in AngularJS quite nicely indeed.
Underscore.js: we could avoid using Underscore.JS, but then our code would be less awesome.

In addition, we bring in 2 CSS features:

Bootstrap: in this app, we use it just for vanilla UI presentation.
Font-Awesome: we love Font Awesome because it lets us use truetype font characters instead of image-based icons. Pretty sweet!

Overall, we were pretty pleased that we could build a nifty UI with so few dependencies. And with that… on to the UI!

The User Interface

Here’s what we intend the UI to look like:

The UI is pretty straightforward – everything is inside a div tag that is managed by a single controller that we’ll set up in the AngularJS code.

<div class="container" ng-app="PubNubAngularApp" ng-controller="MyTextCtrl">
<pre>
NOTE: make sure to update the PubNub keys below with your keys,
and ensure that the sentiment analysis BLOCK is configured properly!
</pre>
<h3>MyText Sentiment Analysis</h3>

We provide a simple text input for a message to send to the PubNub channel as well as a button to perform the publish() action.

<input ng-model="toSend" />
<input type="button" ng-click="publish()" value="Send!" />

Our UI consists of a simple list of messages. We iterate over the messages in the controller scope using a trusty ng-repeat. Each message includes the original text as well as the sentiment analysis emoticon and numeric score (if applicable). The emote() function returns a Font Awesome icon, and the to_trusted() function blesses the string so that AngularJS will display it as HTML.

<ul>
  <li ng-repeat="message in messages track by $index">
    text: {{message.text}}
    <br />
    sentiment: <span ng-bind-html="to_trusted(emote(message.sentiment_type))"></span> (score: {{message.sentiment_score}})
  </li>
</ul>
</div>

And that’s it – a functioning real-time UI in just a handful of code (thanks, AngularJS)!

The AngularJS Code

Now we’re ready to dive into the AngularJS code. It’s not a ton of JavaScript, so this should hopefully be pretty straightforward.

The first lines we encounter set up our application (with a necessary dependency on the PubNub AngularJS service) and a single controller (which we dub MyTextCtrl). Both of these values correspond to the ng-app and ng-controller attributes from the preceding UI code. Note: in this application, we also use the AngularJS $sce service to produce sanitized HTML.

<script>
angular.module('PubNubAngularApp', ["pubnub.angular.service"])
.controller('MyTextCtrl', function($rootScope, $scope, $sce, Pubnub) {

Next up, we initialize a bunch of values. First is an array of message objects which starts out empty. After that, we set up the msgChannel as the channel name where we will send and receive real-time structured data messages.

NOTE: make sure this matches the channel specified by your BLOCK configuration!

  $scope.messages     = [];
  $scope.msgChannel   = 'group-sentiment-analysis-channel';

We initialize the Pubnub object with our PubNub publish and subscribe keys mentioned above, and set a scope variable to make sure the initialization only occurs once.

NOTE: this uses the v3 API syntax.

  if (!$rootScope.initialized) {
    Pubnub.init({
      publish_key: 'YOUR_PUB_KEY',
      subscribe_key: 'YOUR_SUB_KEY',
      ssl:true
    });
    $rootScope.initialized = true;
  }

The next thing we’ll need is a real-time message callback called msgCallback; it takes care of all the real-time messages we need to handle from PubNub. In our case, we have only one scenario – an incoming message containing text with its sentiment analysis. The push() operation should be in a $scope.$apply() call so that AngularJS gets the idea that a change came in asynchronously.

  var msgCallback = function(payload) {
    $scope.$apply(function() {
      $scope.messages.push(payload);
    });
  };

The publish() function takes the contents of the text input, publishes it as a structured data object to the PubNub channel, and resets the text box to empty.

  $scope.publish = function() {
    Pubnub.publish({
      channel: $scope.msgChannel,
      message: {text:$scope.toSend}
    });
    $scope.toSend = "";
  };

We also create an emote() function that translates a message with sentiment analysis decoration into an emoticon.

$scope.emote = function(type) {
  if (type === 'positive') {
    return '<i class="fa fa-smile-o" aria-hidden="true"></i>';
  } else if (type === 'negative') {
    return '<i class="fa fa-frown-o" aria-hidden="true"></i>';
  } else {
    return '<i class="fa fa-meh-o" aria-hidden="true"></i>';
  }
};

We also need a sanitize function (we’ll call it to_trusted()) so that AngularJS will trust the HTML we’re sending to the UI.

$scope.to_trusted = function(html_code) {
  return $sce.trustAsHtml(html_code);
};

Finally, in the main body of the controller, we subscribe() to the message channel (using the JavaScript v3 API syntax) and bind the events to the callback function we just created.

Pubnub.subscribe({ channel: [$scope.msgChannel], message: msgCallback });

We mustn’t forget close out the HTML tags accordingly.

});
</script>
</body>
</html>

Not too shabby for about eighty lines of HTML & JavaScript. And that’s it! You have a working application that automatically analyzes and streams sentiment on a per-output basis.

Like this tutorial? See what else you can do with IBM Watson and BLOCKS.

Analyze User Sentiment and Emotion in Real time: IBM Watson