2

Edit: I have found a closely related question: StackOverflow

This question is not about the differences between functional and acceptance tests! Almost all the info I could find on the web just explains the difference between them. I know that functional tests (FT's) address fringe conditions and bug scenarios, whereas Acceptance Tests (AT's) address business requirements. I achieve both of with SpecFlow.

I am having some trouble wrapping my head around the separation of the two, from a project structure/hierarchy perspective. Currently, I have one unit-test project with an AcceptanceTests folder, and a FunctionalTests folder. All my step definitions are jumbled together in a StepDefinitions folder.

I find I'm having to repeat myself a lot and that the MsTest pane just mixes everything together when I group on Traits. I want to establish what the industry standard is out there, so I have five questions:

  • Do I repeat the "In order to... as a... I want to..." story in a separate feature-file for AT's and FT's?

  • Do I repeat all the AT's scenarios in the FT's as well, or only fringe condition scenarios?

  • Should I keep the AT's and the FT's in their own namespaces and/or their own projects?

  • Should I try to call the FT's scenario step methods from the AT's step definitions, seeing as the grunt-work is being done by the FT's anyway?

  • Any advice about my current setup (e.g. is it overkill to do both?) is welcome.

Community
  • 1
Riegardt Steyn
  • 123
  • 6

1 Answers1

4

I don't know if I agree with your premise. And I think you are kind of conflating a bunch of ideas and terms.

functional testing is any testing that tests the functions of your system. this may be unit (where we are only testing a specific class and mocking/faking dependencies or integration (which is black box testing and tests the interaction between all the running pieces of your system). this is not just limited to fringe conditions and bug scenarios. you should be functionally testing all parts of your application.

acceptance testing is testing performed for/by your business stakeholders that the software meets their acceptance criteria. acceptance tests may be your integration tests, but they should never be unit tests because unit tests do not exercise the entire system.

with that out of the way let's move to SpecFlow. SpecFlow is a nice tool that allows you to create cucumber syntax acceptance tests in a .net.

So let's take an example

First define a feature : Create a New Account

In order to track people who come to my site as a site owner i would like users to create accounts.

Define all the scenarios you are likely to encounter when your are creating a new user.

Scenario Navigate to Account Creation Page:

Given I am logged in as 'Site owner'  
When I have navigate to the site creation page  
Then I see the following fields  
| Field    |
| Username |
| Password | 
And the required fields are marked with '*' 
| Field    | 
| Username |
| Password |

Scenario Create account - Happy: 
Given I am logged in as 'Site owner'
And I have navigated to the site creation page
When I fill out the form as follows
| Username  | Password       |
|'somename' | 'somepassword '|
And I click the 'save' button
Then I should see the 'confirmation' message 'Account created'



Scenario 3: Required fields Missing

Given I am logged in as 'Site owner'
And I have navigated to the site creation page
When I fill out the form as follows
|Username |Password |
|''       | ''      |
And I click the 'save' button
Then I should see the 'error' message 'some error message'

One of the key pieces of SpecFlow is that step definitions are global so use parameters or tables so that you can reuse steps like seeing a confirmation/error message, or logging in, or filling out a form. it gives you a lot of flexibility in how you define your steps

Lunivore
  • 4,232
  • 1
  • 18
  • 23
Fran
  • 859
  • 1
  • 6
  • 8
  • Thank you for your response, Fran. Alas, I already know all of that, and I already have Scenarios set up and passing, both at high business level, and at a lower example-based level. The way I understand it, the high business-level stuff should not have particular numerical examples; it defeats the purpose of being abstract and understandable to a business person. My question goes beyond the basic understanding of what SpecFlow does. SpecFlow is perfect for functional testing because of its support for tables/examples. It is also great for user stories. – Riegardt Steyn Feb 07 '16 at 10:57
  • Unit tests, I only use to prove a method works. Speaking of which, do you jumble them into the test project too, or into yet another assembly? I do know that integration tests should absolutely be in their own assembly, due to running time and resource cost. – Riegardt Steyn Feb 07 '16 at 11:05
  • 1
    I create separate projects for unit test and acceptance tests. 1. Because they are testing separate things and 2. Because the dependencies are different. my units tests are going to have separate dependencies. Units tests are going to be dependent on nunit and some faking tool like moq or fakeiteasy. My acceptance tests are dependent on nunit, specflow, selenium webdriver. – Fran Feb 07 '16 at 12:39