Are RSpec and Cucumber really worth it?

Question

I know most RoR programmers are testing addicts and I understand the advantages of a large testing suite but when I start testing, I never get such a large suite and I always wonder "Am I testing the right way? Are there really efficient?". I'm often dealing with integration tests testing only the way the application behave.

First, is testing really worth it? I mean, is the time spent on writing tests really worth it?

Then, I use RSpec, I've recently discovered Cucumber, used it for a while but I don't know if writing all these steps is really worth the trouble? I know I can reuse steps but I never know if these steps are too complete or not: For instance, I've been using a Given I am logged in as (.+) but I don't know if I must say in its definition Given there's a user called $1 because it can duplicate the user if ever created but that's not worthwhile always having a step before Given I am logged in as (.+). It's quite a lot of code that will maybe rarely be useful. I guess there are not new bugs on the parts tested every day... So is Cucumber really worthwhile compared to RSpec?

Answer 1

My 'ah-ha!' moments about testing in Ruby and Rails came when I really sat down and read the definitive resources on the subject, the Rspec and Cucumber books. I shared your initial disdain of Cucumber, but then I realized that I was looking at the picture from quite the wrong angle.

Basically, Cucumber is about BDD (behaviour driven development) - you use Cucumber to plan your features, what you're going to work on next. Hmm, next you want users to be able to promote posts on a forum or something (to steal an example ;)) So you write something simple.

Given I am logged in
And I can see the post "BDD is awesome"
When I vote the post up
Then the post should have one more vote
And the page should show a message thanking me for my vote.

Note that there's no references to anything code related in there pretty much. That comes in your steps. When you refactor your code, you might have to change your step definitions, but the behaviour (your feature) will never need to change.

Now every time you run your Cucumber feature, you'll pretty much be led through how to test the feature using TDD (test driven development). This is done at a lower level using RSpec.

First run - my first step definition is undefined. Copy the block to define it in say user_steps.rb or even session_steps.rb because it relates to users and their sessions. Now, how do you define that a user is logged in? You can take them through the login process.

Given /^I am logged in$/ do
  visit login_path
  fill_in :name, :with => 'Joe'
  fill_in :password, :with => 'Password'
  click_button 'submit'
end

Should be all happy. Second step.

Given /^I can see the post "(.+)"$/ do |name|
  visit post_path(Post.find_by_name(name))
end

Again pretty easy. Note that if we totally redo our login process, or how our posts are defined and shown, we don't have to change the behaviour. Third step.

When /^I vote the post up$/ do
  pending 
end 

Here's where you're starting to talk about new functionality, but you don't quite know how it's going to work yet. How do you vote a post up? You might click an image of a +1 or something, which does an ajax post to a controller, which returns JSON, or some such. So now you can move into pure Rspec testing.

• Test your view to make sure the +1 image is displayed,
• Test your controller that it behaves correctly when it receives a given ajax request of the right format (both the happy and unhappy paths - what if an invalid post ID is received? What happens if the user has used up their 25 upvotes in a day? Does it increment the number of votes correctly?)
• Test your javascript that it responds correctly when given a blob of JSON in the right format (does it update the +1 image to show it's been used? (thinking of Google+ here...) Does it show the thank you message? etc.)

All of this doesn't affect the behaviour - but when you're finished dealing with the lower level testing, it will be trivial to fill in the step definition for how to vote a post up. It might be as simple as click_link '+1'. And the rest of the steps are testing results, which again should be straightforward to do. And when you're done, then you know your feature is complete and finished. If the necessary behaviour changes, you can tweak your feature, else you can tweak your implementation code in perfect safety.

I hope this makes sense. It's all been off the top of my head, but I think it demonstrates the difference between BDD and TDD, and why Cucumber and RSpec serve different needs.

Answer 2

Testing, in my opinion, is an art. Doing TDD (using RSpec or any other framework) initially feels like you're "wasting your time." This is understandable because you are not writing any production code.

However, you start seeing the benefit of TDD when you need to enhance your codebase while ensuring that everything else still works. TDD helps you catch regression errors as early as possible. Doing this has saved me days of work because I had focused tests which pointed out my errors.

Furthermore, having tests can be beneficial for code reviews because your reviewer can see what scenarios you are testing and how your code is meant to be used.

Once you get into the swing of TDD, doing anything else feels wrong.

Answer 3

My take is that you are right on the Cucumber front. Writing all those steps is a lot of trouble, and the benefits don't justify the pain. I've written extensively about the six disadvantages of using Cucumber here: Why Bother With Cucumber Testing?

Unit tests and regular integration tests, done either with Rspec or Test::Unit do make a lot of sense, but luckily these are far faster to write than Cucumber tests. For one you can use pure Ruby instead of having to fight Gherkin's wordy and awkward syntax.

Answer 4

What I personally believe is that RSpec testing is a definite must. Let's say for example that you want to write a new feature, and which also has reference to some other feature, and that feature might be referenced with some other module or methods. So how can you make sure that what you are writing is not breaking any other part of the application?

Assume that you have a big application and that you have coded something trivial compared to the overall application, will you retest the entire application by clicking every link in the application to make sure it works everytime you change a single line of code?

However, I believe that Cucumber testing is not a must. I think that integration testing using RSpec itself makes more sense until and unless you have to get tests checked by your client. Which in my experience is RARE. If your team consists entirely of developers then I think that you should rather replace the Cucumber steps for RSpec feature testing. And I think after the RSpec 3 DSL, the tests are pretty much readable.

Ex :

Cucumber Step Definition :

Given /^I am logged in$/ do
  visit login_path
  fill_in :name, :with => 'Joe'
  fill_in :password, :with => 'Password'
  click_button 'submit'
end

RSpec Feature test :

feature 'Given the user is logged in' do
      visit login_path
      fill_in :name, :with => 'Joe'
      fill_in :password, :with => 'Password'
      click_link_or_button 'submit'
end

I think that rather than having Cucumber features, RSpec features do the same thing without the extra headache of writing another step definitions.

Other than that it is also purely your own preference.

Hope this could help you understand a little bit.

Answer 5

In my opinion the first thing its to differentiate between practices and concrete frameworks. Cucumber its not BDD, RSpec its not TDD.

If you want to test your system RSpec its a good tool, you can do TDD or BDD with RSpec, in fact TDD and BDD are the same thing. Someone says "BDD its TDD done right" and i'm totally agree with that, BDD primarily its about testing features/behaviors instead of testing methods/classes. In fact the TDD that Kent Beck describes its about features, but BDD help a lot of people in understand this key difference and its a great contribution from Dan North to the development community.

Use Cucumber if you feel that you need a better tool to communicate with business people, for example if cucumber allow that your business people or Product owner helps the team in writing or revising scenarios. Other people like cucumber because this scenarios are a really good live-documentation of a system, if you feel that you need this type of documentation try cucumber.

In Summary:

• If you want to do TDD/BDD yourself or you team -> try RSpec
• If you want a better way to communicate with business with User Histories and Scenarios -> try cucumber
• If you want live documentation of your system features -> try cucumber.

Of course the last two have a hight cost associated, you need to evaluate if you really need that and worth the effort, this off course depends totally on your project and your environment and the decision its up to you.

But always remember that RSpec and Cucumber are only tools, and tools solve concrete problems, what problem you want to solve?, ask yourself this question and you probably are in a better position to select the right tool. Be a better programmer its about taking this decisions not about using X or Y framework/tool/library/technology.