0

I've seen some pretty darned restrictive 'chat-systems' in kids games, that make it nigh-on-impossible to get much of a message across whilst typing.

So, I looked at some ideas, and there where two main ones, the whitelist and the blacklist:

  • Whitelisted words are the only words you can use.
    • Certainly safe language without a doubt.
    • Very, very restricted conversation.
    • Large data needed for freer conversation.
  • Blacklisted words are the only words you can't use.
    • Easy detection of specific words.
    • May not be a comprehensive list.
    • Doesn't deal with 5tr4n9e w4y5 0f typ1n9.

Clearly, the whitelist is not the way to go for easy, free conversation, but the blacklist could easily be bypassed.

Furthermore, how should disallowed words be dealt with? For instance, If I were to send the string "You're truly elucidating!" through the whitelist, which probably wouldn't have a benign but very complex word such as elucidate in it's database, and it instead broadcast "You're truly ***********!, I doubt many people would take it as a compliment. The blacklist has a similar problem - should I blank out curse words, or just entirely prevent such a message fro being sent?

So: What method should I use to detect and handle obcene language in user input without restricting conversation, but keeping things 100% safe?

As a sidenote, an API for a dictionary that marks out obscene language would be useful for both a whitelist and a blacklist.

AJF
  • 211
  • 2
  • 8
  • 2
    This http://habitatchronicles.com/2007/03/the-untold-history-of-toontowns-speedchat-or-blockchattm-from-disney-finally-arrives/ might be an interesting read as to why whitelists don't work. In short how do you prevent a 14 year old from comming up with a sentance such as "I want to stick my long-necked Giraffe up your fluffy white bunny. "? – Thijser May 17 '15 at 18:01
  • "What method should I use to detect and handle obcene language in user input without restricting conversation". Sounds like a contradiction to me. You want to restrict conversation without restricting it. – Andy May 17 '15 at 18:47
  • @Andy Haha, true! Maybe I should say restricting *polite* conversation. – AJF May 17 '15 at 18:49

2 Answers2

7

There is no technical solution to this problem.

From Jeff Atwood's article:

I'm doubtful it will ever be possible to solve this particular problem through code alone.

If a user really wants to use bad words in messages, he will still achieve it if the only filter is a program.

  • Is “fuck” a forbidden word? OK, we'll try “f*uck”: still understandable, but more difficult to catch for an application.

  • Still caught? What about “uck (the first letter is not an “f”, but the mathematical sans-serif small “f” 0x1d5bf).

  • What? Your app understands that 0x1d5bf character is visually similar to the character “f”? It's OK, there are plenty of other characters in Unicode which will appear visually similar.

Even a more basic problem—determining what constitutes a word—is not that easy. In a perfect ASCII world, a word would be some characters in [A-Za-z] range separated by a space. In a real world, you have punctuation, diacritics, characters which are not displayed, and dozens of other Unicode things which can make a simple task of separating text into words in the same way a human will do very, very challenging.

Of course, all those problems are solved with a dictionary of whitelisted words. You're back to ASCII, space character and basic punctuation. In such restricted context, it becomes easy to parse text programmatically and reject text which contains something your app doesn't understand. This, in turn, has two major problems:

  • Nobody sane will enjoy a conversation in a such controlled environment.
  • Obscenity will take other routes. It's always possible to tell obscene things without using obscene words.

Instead, why not using a system similar to Stack Exchange's one with the ability for the community to edit messages, to close ones for which the site is not the right place, and flag ones which are particularly problematic or harmful?

Arseni Mourzenko
  • 134,780
  • 31
  • 343
  • 513
  • 1
    +1 for suggesting community moderation. I dunno how likely kids are to make use of it, but it is the only real solution to this problem. – Ixrec May 17 '15 at 15:28
1

I've heard great things about WebPurify's profanity APIs. It's not free, but they have a thing on their website where you can type sample text in and see if it detects anything:

http://www.webpurify.com/features/

I just put in f-u-c-k and it detected profanity, so that's a good sign...

Hank
  • 11
  • 1