Build

Detect and Filter Profanity in a Real-time Chat App

3 min read • Michael Carroll on Nov 16, 2016
Detect and Filter Profanity in a Real-time Chat App.jpg

We’d like to introduce you to one of our BLOCKS partners, Neutrino API. Neutrino provides a variety of useful web services and tools for web developers, like phone and email validate, image resizing, IP geolocation, and in this tutorial’s case – detecting and filtering profanity. We’ll build a real-time chat app, and using the Neutrino API BLOCK, we’ll detect and filter profanity, and block future messages from those specific IP addresses.

Getting Started with Neutrino API and PubNub BLOCKS

We’ve created a template showing how to ban users in a chat session that are using profanity by combining Neutrino API with the BLOCKS KV Store. You’ll need the Neutrino API BLOCK for IP address blocking. Just click the ‘Try it Now’ button to make a copy of it in your PubNub account. If you don’t have a PubNub account yet, you’ll need to sign up here.

Next, go to the Neutrino API signup page to create your free account. After you create the account, pick a username and generate an API key. Immediately save this API key somewhere secure. Now go to the PubNub BLOCK you just created and put in your keys.

const userId = 'my-username'
const apiKey = 'my-long-api-key;

Save this BLOCK then press the start button in the upper right corner. Once the BLOCK starts you can send a test payload using the JSON editor in the lower left corner.

Try sending this test payload:

{
  "text": "You are cool"
}

Your message goes through fine. Now try sending this test payload:

{
  "text": "You are poop"
}

Now the IP address is blocked.

How It Works

Every PubNub message contains the IP address of the sender in the request.meta.clientip property. This template stores a list of blocked IP addresses in the KV store. First, it pulls out the data with the call to store.get(), then checks if the IP is already in the data store. If the address is already there then it will block the message by removing the message text and putting in the reason it was blocked.

return store.get('blocked_ip_data').then(blockedIPData => {
  blockedIPData = blockedIPData || {};
  // extract client ip address from request
  const clientip = request.meta.clientip;
  // If client IP address is in blocked data, set message text to
  // null and add ip_status key, with value having details of
  // blocking reason i.e. uuid and bad words detected
  if (clientip in blockedIPData) {
    request.message.text = null;
    request.message.ip_status = {
    blocked: true,
    data: blockedIPData[clientip],
    ip: clientip
  };
  return request;
}

If the user isn’t already blocked, then it sends the message text to the Neutrino API with a fetch call:

const queryParams = {
  'user-id': userId,
  'api-key': apiKey,
  content: request.message.text,
  'censor-character': '*'
};
// This is a before publish event handler and we can't return
// request before modifying it, so we have to make a synchronous get
// request. This is done using .done() at the end
// Please note that in all before publish handlers, where you wanna
// modify the message based on content from some external source,
// you need to make synchronous requests since we can't return
// request before modifying it.
console.log("doing the call");
return xhr.fetch(apiUrl + '?' + query.stringify(queryParams))

If the response indicates that the user has used a bad word, then the user’s IP is added to the blocked list, otherwise, the message is sent on as normal.

.then(r => {
  console.log(r);
  const body = JSON.parse(r.body);
  if (('is-bad' in body) && body['is-bad']) {
  let badWords = body['bad-words-list'];
  // If message text contains bad words, set message text
  // to null and add ip_status key, with value having
  // details of blocking reason i.e. uuid and bad words
  // detected. Also add this information to store
  blockedIPData[clientip] = {
    uuid: request.params.uuid,
    bad_words: badWords,
    text: request.message.text
  };
  request.message.text = null;
  request.message.ip_status = {
    blocked: true,
    data: blockedIPData[clientip],
    ip: clientip
  };
  console.log(blockedIPData);
  store.set('blocked_ip_data', blockedIPData);
}
return request;

That’s all there is too it. Filtering profanity is just one of the many interesting Neutrino APIs you can call from BLOCKS. They also provide phone number validation, HTML cleanup, unit conversions, SMS verification, QR code generation, and many more.

Pro tip: you can use a counter from the KV store to track how many times a user has used a bad word, providing them with a warning before blocking them entirely.

0