2

I am using ColdFusion 8 and jQuery 1.7.

** This is a programming question, because the solution I am questioning requires programming. It may not be the right solution to the problem, but if it is, then I need to figure out how to best program the concept. **

When a user comes to our site, we track their session by writing various CGI variables to a database using a CFC and stored procures. First we filter out non human traffic by keywords in the user agent such as "bot". Unfortunately a lot of bots and spammers mask their user agents. Later, we try to exclude from our visitor reports the bad bots and a few other known entities that are scraping pages and such. But this is a manual process.

We are considering using an additional/alternate method of tracking usage. Once the user's page loads, we will use JavaScript to send the CGI variables from the client back to our server and store them. Specifically, we'll write the server variables to JavaScript on each page and then have JavaScript send them right back to us. If a bot or user doesn't fully view the page or have JavaScript enabled, the usage won't be counted is a real user.

Correct me if I am wrong, but this is the same method that Google Analytics uses to track user behavior.

Our goal is to eliminate good and bad bots from being counted as visitors in our reports. Does using JavaScript on a page like this minimize bots being counted? Is there a gaping hole in this plan?

Evik James
  • 178
  • 6
  • I can tell you that neither my husband and I have javascript running by default. With how popular No-script and many other such add-ons are, I think you might be getting more false negatives using this practice. Will not counting these users cause more issues in your reports than counting a few bots as users is something your team needs to decide. – Tyanna Feb 21 '12 at 21:46
  • How do you plan to get around bots that are just automated instances of IE? – Bryan Boettcher Feb 21 '12 at 21:49
  • @Tyanna, what benefit is there to browsing with JavaScript disabled? Is it safer? Is it because of performance issues on your computer or security? – Evik James Feb 21 '12 at 21:54
  • There's a myriad of web log analyzing tools that have long since solved this problem. Can you use one of them instead? I'm all for doing your own thing, but thats such a specialized area that needs to be constantly kept up with, it might be better to use a canned approach here. – GrandmasterB Feb 22 '12 at 04:41

1 Answers1

3

I have implemented a few tracking features and took the following approach. Use an image tag that renders a dynamic file image. Make the image a dynamic call that you can then track using whatever mechanism your backend provides (ColdFusion in this case). Again it's not perfect, but most bots don't download your images etc.

Your dynamic call just returns a 0 byte image.

You can then capture everything that is available normally in the request, including the refer which is the page your image is embedded on. You can even put query parameters on the image tag if you want to capture additional information about where the image tag is listed.

Bill Leeper
  • 4,113
  • 15
  • 20