2

Our company is using an external API that is actually in beta state. This means it's not stable at all yet, and it changes its requests/responses every week or so.

I'd like to write test to ensure my code is working. Unit tests are already done for my own code. But the API itself isn't tested, because it would violate the first rule of FIRST (Fast tests). If I test the API requests/responses 'live' (they have a sandbox version of the API), tests take around 7 seconds for themselves. That's a long time.

How could I be sure the API still uses the same structure for requests and responses in this case, is it a special case where slow tests are ok ? My hierarchy wants to use this API asap obviously, and I don't want any bug whatsoever.

Steve Chamaillard
  • 1,691
  • 1
  • 11
  • 20
  • 1
    unless you name the api we will have to assume the only way to check for breaking changes is to call it. But they might have a blog or a version check or something – Ewan Sep 21 '16 at 11:50
  • I don't think they have a version check but that could be a pretty smart idea to ask for a callback URL to test if the API ever changes ! That would be a lot faster. – Steve Chamaillard Sep 21 '16 at 12:01

2 Answers2

1

Honestly, the API you're calling should have a versioning scheme. IE, an API i've worked with in the past was formatted http://clientaddress.com/v1/endpoints, but they were in testing with a v2 that lived at http://clientaddress.com/v2/endpoints. While we wanted to use the new API asap, we could only test against v1 (the stable version) to ensure things were working. That said, we went through the trouble of overloading methods/calls/whatever to accept the new objects from the v2 api, then call into our v1-formatted methods using these objects. This way, we were able to test against an 'incomplete' product by making sure the logic on our end was correct.

The other option, albeit kind of 'cheating' since (I think?) you don't ever want to test with mocks, is to create a dummy API locally that returns a pre-set object or objects in the format you're expecting from the client API, and have this method be what is called whenever you want to talk to the client API. This way, you can prove your code works with fake objects, and when you're ready to test with the client api in its final form, you can modify this one method to go to the real thing with minimal work.

Adam Wells
  • 908
  • 4
  • 15
  • Thanks for the answer, the API uses a versioning scheme like the one you describe but they don't change v1 to vX every time they make a change since it happens very often. I can't use the second method because every call to their API is inserted as data that I'll use later in their database. – Steve Chamaillard Sep 21 '16 at 14:16
  • The second option is exclusively for testing. I wouldn't ever want to run right into my database during testing, I just want to make sure my code works. Have you considered Rhino Mocks or some similiar library for that? Pretty much, i'd be interested in the code working up to the insert to database point because I know my database is (probably) working as intended. – Adam Wells Sep 21 '16 at 14:57
1

I'd like to write test to ensure my code is working. Unit tests are already done for my own code. But the API itself isn't tested, because it would violate the first rule of FIRST (Fast tests). If I test the API requests/responses 'live' (they have a sandbox version of the API), tests take around 7 seconds for themselves. That's a long time.

I completely agree that adding 7 seconds to your unit tests is not ideal - but even if it was 7 microseconds you still shouldn't test the API with unit tests in your code. You don't need to test the API every time you make a change to your code - and you do need to test it even if you don't make changes to your code for a month: you need to test it every time they make changes to their code.

Of course, you don't know how often they do that, so I'd suggest running the test automatically once an hour/day/week, depending on how often you expect changes.

Of course, whether it's a good idea to actually have those tests is debatable. Personally, I believe that having some sanity checks makes sense, especially for unstable services where someone may make a change without even changing the patch version and then you have to investigate for hours till you realise it - but I digress: even assuming best practices, you might have misinterpreted the API and a change that wouldn't have affected normal use of it breaks your code.

Thanos Tintinidis
  • 721
  • 3
  • 10