8

I am planing to build a website which one of its main features is the usage of pictures. I was wondering what existing techniques are used to prevent offensive pictures (i.e pornography) to get loaded by users. i.e What does Facebook or Pinterest use?

As well I would like to know what your recommendations would be to control offensive pictures to get uploaded to the site (OpenSource tools maybe..) Taking into consideration this is a personal project and will be developed with a very small budget.

Thank you

Andreas Siegers
  • 81
  • 4
  • 1
    Nice comment from an [existing question](http://programmers.stackexchange.com/questions/54155/a-client-wants-us-to-screen-work-machines-for-pornography-is-it-possible): 'Create a script that posts all images it finds on 4chan; if other members answer "MOAR!", you know it's porn. If the script gets banned, it's probably CP.' – user281377 – Corbin March Jun 27 '12 at 20:07
  • You could develop a CAPTCHA where the user is challenged to pick out the non-pornographic image from a gallery (or vice-versa). – emory Jun 27 '12 at 22:09

6 Answers6

14

There is no technological solution that will tell you that an image is "offensive".

You need to consider that "offensive" is culture dependent (as well as an individual thing) - that is, what you may find offensive and what I may are probably very different.

This is a social issue and can only be solved with social solutions - this may mean moderation (or community moderation).

So - either get moderators to approve/reject each image or put a system in place that allows your users to flag images as offensive and that will remove such images when a certain threshold has been reached. You may want to track users who upload such images in order to moderate/ban them as well.

Oded
  • 53,326
  • 19
  • 166
  • 181
  • 1
    Pretty much this, though I thought there were tools that can tell what % of a picture was skin that could help filter pornography etc, but have never had first hand experience myself. – Anonymous Jun 27 '12 at 18:45
  • Oded is absolutely right. Even when you register somewhere they demand you to type some symbols from an image, which might not seem so complicated to a human but is almost impossible to decipher for a bot. – superM Jun 27 '12 at 18:47
  • @Anonymous- Sure, but that's one specific scenario. What about tools that detect violence in an image? Or images portraying other nasties? – Oded Jun 27 '12 at 18:47
  • @Anonymous, the paintings of Renaissance, for example, contain pretty much naked skin. But they cannot be considered nor pornography neither offensive. – superM Jun 27 '12 at 18:48
  • 5
    +1. This is really the only answer. What you need is a flag/"report offensive content" mechanism, not a filtering algorithm. – Mason Wheeler Jun 27 '12 at 18:48
  • 1
    @Anonymous- Check out all these totally pornographic results I found on google image search! https://www.google.com/search?&q=face – octern Jun 27 '12 at 18:53
  • 5
    @superM I believe you underestimate people's ability to be offended. – Philip Jun 27 '12 at 18:58
  • @Philip, Maybe. I remember that story in NY when they forbade teachers to say words like mother, dinosaur, etc at schools! Its sometime absurd. – superM Jun 27 '12 at 19:02
  • I'd say you could safely assume that removing porn images would be appropriate.. and there are servcies / libs out there that do this – hanzolo Jun 27 '12 at 19:04
  • Mechanical Turk perhaps – Kevin Jun 27 '12 at 19:13
  • @Oded, superM, Mason Wheeler, octern - It was just a single example. It would make sense to use this kind of censorship in schools, for example. I was just trying to think of other ways potentially offensive content can be filtered. – Anonymous Jun 27 '12 at 21:31
  • @JohnRobertson - The original post did not include that bit of information. – Oded Jun 28 '12 at 15:26
  • 1
    @Anonymous That approach has been tried http://en.wikipedia.org/wiki/Green_Dam_Youth_Escort#Functional_defects. The PRC is working on this. – emory Jun 29 '12 at 20:42
3

There are Image Content Filters out there which can prevent this type of thing. They turn the image contrast up and trace the edges and compare with "known" adult images.. There's a lot of research on the subject.. and i know that there is at least 1 service out there that does it, it escapes me for now.. but they're out there

hanzolo
  • 3,111
  • 19
  • 21
  • 2
    What about violent imagery? – Oded Jun 27 '12 at 19:04
  • 1
    i'm not sure what the bounds of the image detectors where.. however, if it's doing a loose match on known images of violence.. it could probably get most of those too.. the problem will be in "false - positives" – hanzolo Jun 27 '12 at 19:06
2

Some attempts at this exist as a concept against specific kinds of offensive based on edge detection. I have seen research papers on it, but as a practical matter whatever you would be "detecting" would easily be gameable by tinting or anything that breaks up the obvious outlines of the objects.

Even if you could get a working implementation of this the internet would just upload you some smut with WWII navy style dazzle paint on the performers or performers in ghillie suits just to be difficult.

If you don't mean offensive as smut but as violent or off-color I don't even think you could come up with an automated way to rate it even if you had a perfect image detection process just due to the problem of nobody agrees when it crosses the line.

Bill
  • 8,330
  • 24
  • 52
1

It all comes back to pattern recognition. The reason we have facial recognition software is faces, for the most part, have similar traits, and most of the pictures we take of people have them facing the camera. But that was still very difficult to develop. Let us consider for a moment a picture of bomb striking a populated target. How would you create a pattern for that? The picture would have an explosion, fairly simple for a human to figure out, but rather hard to program for. An explosion could be of various sizes, shapes and colors. Let's say you create some sort of rule based system. What if the picture is an explosion for avalanche control in the mountains? That is not violent or offensive, but it would contain a lot of the same characteristics.

As many other people have pointed out, it is very hard to define obscene. The US Supreme Court attempted to in the case of Miller vs. California, and as a result we have the so called Miller Test which consists of three parts:

  • Whether "the average person, applying contemporary community standards", would find that the work, taken as a whole, appeals to the prurient interest,
  • Whether the work depicts/describes, in a patently offensive way, sexual conduct specifically defined by applicable state law,
  • Whether the work, taken as a whole, lacks serious literary, artistic, political or scientific value

Nice and ambiguous, makes things a little hard to program for. Most sites that allow you to upload images have some sort of human moderation. For example, all pictures posted to online dating sites have to follow a set of rules and have to be approved manually.

bwalk2895
  • 1,988
  • 1
  • 13
  • 15
1

As others have pointed out there are some technological solutions but they only get you so far. This article from the NY Times discusses how large sites (like Facebook) filter content.

To answer your question: "What does Facebook or Pinterest use?"

People.

NoobsArePeople2
  • 351
  • 1
  • 3
0

Don't forget that once you have broadcast that you filter out "offensive" pictures, you open yourself up to lawsuits if you miss one. You could add a flag button to each page and be prepared to delete the offending photo after reviewing it.

SnoopDougieDoug
  • 1,947
  • 12
  • 8