8

I have a CI server (Hudson) which merrily builds, runs unit tests and deploys to the development environment but I'd now like to get it running the integration tests.

The integration tests will hit a database and that database will be consistently being changed to contain the data relevant to the test in question. This however leads to a problem - how do I make sure the database is not being splatted with data for one test and then that data being override by a second project before the first set of tests complete?

I am current using the "hope" method, which is not working out too badly at the moment, but mostly due to the fact that we only have a small number of integration tests set up on CI.

As I see it I have the following options:

  • Test-local (in memory) databases
    I'm not sure if any in-memory databases handle all the scaryness of Oracles triggers and packages etc, and anything less I don't feel would be a worth while test.
  • CI Executor-local databases
    A fair amount of work would be needed to set this up and keep 'em up to date, but defiantly an option (most of the work is already done to keep the current CI database up-to-date).
  • Single "integration test" executor
    Likely the easiest to implement, but would mean the integration tests could fall quite far behind.
  • Locking the database (or set of tables)

I'm sure I've missed some ways (please add them). How do you run database-based integration tests on the CI server? What issues have you had and what method do you recommend? (Note: While I use Hudson, I'm happy to accept answers for any CI server, the ideas I'm sure will be portable, even if the details are not).

Cheers,
     Mlk

mlk
  • 1,049
  • 8
  • 18
  • 1
    This [stackoverflow question](http://stackoverflow.com/questions/4768682/looking-for-an-in-memory-database-with-same-sql-syntax-as-oracle) may be helpful to you. – Michael K Feb 16 '11 at 15:32

2 Answers2

1

Just an idea, since I've never actually done this, but you could set up a task to get the DDL for the database schema you are testing against, and recreate it either in memory or on another database. It looks like it's fairly simple in Oracle. Then you can automatically generate the new database whenever it changes.

It looks like Oracle has an in-memory database called TimesTen. It might be worth looking at. I would definately prefer to use an in-memory database if possible, since it's usually faster than hard storage.

Michael K
  • 15,539
  • 9
  • 61
  • 93
  • I tried in-memory dbs a few years back but I found it to be much slower than using a real DB (I am happy to give it a go again). The real db exists with the schema (but no content). The in-memory needs to be started and have the schema created for each test/test run. I'll give TimeTen a play and see if I can link it up to our existing database cloning and schema updating system. – mlk Feb 16 '11 at 15:57
  • Maybe you could create a singleton that holds the reference to the database, and just clean out the tables after each test? I guess my experience hasn't been with large enough data sets. – Michael K Feb 16 '11 at 16:09
  • Have used Apache Derby for this, it works. – Tim Williscroft Feb 16 '11 at 20:19
1

We run database-based integration tests by standing up a database server to work with the CI box and host the CI instance. Kind of like how staging and production each have their own database server.

Wyatt Barnett
  • 20,685
  • 50
  • 69
  • How do you handle multiply tests projects hitting the same database at the same time, or do you have a database per executor? – mlk Feb 16 '11 at 15:58
  • 2
    Everyone gets their own database (or databases as required) in SQL server terms, not sure how that translates to Oracle-land. – Wyatt Barnett Feb 16 '11 at 17:10
  • Assuming I'm reading this correctly, each CI executor gets its own database? How do you reconfigure the app.config (/however your database config is done) for each executor? – mlk Feb 16 '11 at 17:31
  • 1
    Again, I'm not sure if I'm understanding your terminology here, but for CI, we set the box up with the same conventions we use in the development team so things just snap into place -- eg, everyone is using the database named "foo" at the sql server localhost\SQLEXPRESS so it just works. – Wyatt Barnett Feb 16 '11 at 17:33
  • Ah so you run the db locally or is SQLEXPRESS an in-memory database? If it is local this limits you to a single executor per CI instance. – mlk Feb 16 '11 at 21:20
  • I still have no idea what an executor is here, but SQL server express is a stripped down version of SQL server I don't have to worry about licensing for. Conveniently, it takes the same T-SQL commands as the full-blown SQL servers we deploy to so it lets us accomplish testing effectively. Note this requires that your app has an idea how to stand up and build it's own database . . .. – Wyatt Barnett Feb 17 '11 at 00:22
  • Thanks. I think you are talking about test-local databases (an executor is simply a thread on a CI node, i.e. you could have a CI slave with 4 CPUs and so have it compile four jobs at the same time). – mlk Feb 17 '11 at 10:34
  • Nope, would really what you call "CI-Executor-local databases" I think -- they are normal database instances, they just get spun up and migrated by the CI process. – Wyatt Barnett Feb 17 '11 at 13:53
  • Cheers. That seams like a good solution. I'll give it a play. – mlk Feb 18 '11 at 10:31