3

I've recently ran into some issues where some reports were not generating the proper results. We already have a QA department that reviews the reports and then our business owners review them as well; this hasn't prevented the issue.

I'm trying to figure out a decent way to have a test harness to ensure reports are giving me the desired result.

So far, I've thought about having a static snapshot of the database (point-in-time), I would then

  1. run the report sprocs (stored procedures) against my data with defined parameters
  2. save off the output
  3. write testing sprocs that call the reporting sprocs with my define parameters
  4. Verify the results matched using the EXCEPT keyword.

Can you please offer some suggestions and alternatives?

FrustratedWithFormsDesigner
  • 46,105
  • 7
  • 126
  • 176
UnhandledExcepSean
  • 153
  • 5
  • When / how are these issues actually being caught if not by QA and/or the business owners who review them? – Mike Feb 20 '14 at 14:09
  • We catch them significantly later than the initial review the business owners perform. Either our business owners see results that don't make sense, or we notice it. Normally the former. :( – UnhandledExcepSean Feb 20 '14 at 14:31
  • Your idea is the right way. Sounds like this question is a case of [Rubber Duck Problem Solving](http://www.codinghorror.com/blog/2012/03/rubber-duck-problem-solving.html) – Jimmy Hoffa Feb 20 '14 at 16:05

1 Answers1

1

It seems to me like you need an automated test.

You need a test data-set which covers all possible data constellations which might or might not be problematic and the desired output which is supposed to be generated from that dataset.

Then you just need a program which automatically runs the test, compares the results against the desired results and reports any discrepancies.

Philipp
  • 23,166
  • 6
  • 61
  • 67
  • This will probably also need input from the end-users about the problems they're experiencing and what possible scenarios might cause other problems in the future. There's also the possibility of bad data somewhere in the data source which might not be picked up by all test cases but is still interfering with reports in production. That could also be related to the situation where the report is developed/tested against a subset of the real data set, so it's not running in the same conditions, and so some scenarios might not be present in the test data. – FrustratedWithFormsDesigner Feb 20 '14 at 14:54