9

A little background of me - I'm a manual tester for almost 2 years within an Agile environment using SCRUM (1-2 weeks sprints). So I'm wanting to introduce automation testing in my work using Selenium WebDriver (with Java).

My question is during when should I test the functionality manually and when should I convert them for automation testing?

I have been reading and getting different approach, such as:

  1. When a new sprint is starting, convert the user stories to automated scripts from the previous sprint, OR;
  2. Convert the user stories within the same sprint.

Any advice/s would be very much appreciated. Thank you in advance.

Jay
  • 101
  • 1
  • 3
  • 4
    Please don't cross post the same question over two different stack exchange sites. Please delete one of them. – R Sahu May 01 '17 at 06:25
  • 2
    Cross posted at https://sqa.stackexchange.com/questions/27017/which-stage-of-agile-scrum-should-we-start-creating-automation-tests. – R Sahu May 01 '17 at 06:26

6 Answers6

13

Test automation (and all other testing) should be part of the definition of done. This in order to make a potentially shippable product. Can you ship if it wasn't tested?

Testing should also be a whole team approach, so test-automation is not the testers responsibility. Start thinking about testing as soon as possible in the process.

Test automation is so important in Agile because:

Organizational Agility is constrained by Technical Agility

In other words, when you are slow in making changes to your product, then it doesn’t matter how you structure your teams, your organization or what framework you adopt, you will be slow to respond to changes.

https://less.works/less/technical-excellence/index.html

If you postpone testing until another iteration you will always be lagging behind. Making it harder to change the direction of the product as it is harder to refactor and safe-guard the external behavior of the product. Having any repetitive manual testing is key in slowing you down, automate-it!

A lot of testers will tell you that you should not start testing end-to-end until the product interface has stabilized. Don't wait, instead make good use of PageObjects and make sure your tests are maintainable and make it a developer responsibility to create and fix them.

Niels van Reijmersdal
  • 753
  • 3
  • 12
  • I disagree on the first "should". The definition of done is something that the Scrum team needs to work out for itself. If the team considers automated testing important, then they will include it as part of their definition. However the process itself doesn't say that they must, or even that they should. It's something that's entirely up to the team, and there is no right, wrong, or preferred answer. – aroth May 01 '17 at 13:17
  • @aroth I agree with you, but in nearly all the cases you build a larger software product you want to add testing to the DoD. Therefor I think it is good to tell people you should, at least seriously think about it. As a tester I believe testing at the end by a separate team is the first steps to Wagile. But yes there are situations and or cases where testing might not even be needed. – Niels van Reijmersdal May 01 '17 at 13:50
2

The key thing is that you not mark a story complete unless you have written automated tests for that story.

So 1 seems to be out, as you are writing tests for a task completed in a previous sprint. what if the tests fail?

Ewan
  • 70,664
  • 5
  • 76
  • 161
  • So if in week one of a new sprint no user story of that sprint is in a state it can be regression tested, you suggest the OP should go home and do nothing? Does not sound very efficient to me ;-) – Doc Brown May 01 '17 at 09:06
  • Tester should use that first week to question the spec "hmmmm as a user I would expect background music on my web page.." find bugs in incomplete stories and generally make trouble. Devs are allowed to say they cant start untill test plans are written – Ewan May 01 '17 at 11:19
  • @DocBrown: in week one of a new sprint the tester has an incredible amount of work to do. They need to understand what the developers are building by working with the product owner and the developers. They need to work with the developer to make sure they make the code testable. They can begin to work on an automated test plan. They can even begin to write some tests. If you have a proper test framework where your tests are written at a high level of abstraction, you can write them before the code is ready. – Bryan Oakley May 01 '17 at 12:34
1

Ideally you should write automated tests in the same sprint that the code is written. Code shouldn't be considered "done" until automated tests have been written, and you must get code to a "done" state by the end of a sprint.

You should start the process on the first day of the sprint by working with the developer to understand the code, and to help them understand your needs as a tester. For example, if you are building web pages you can help them understand the need to add unique identifiers for every page element that you need to interact with.

Remember that in scrum, your job isn't to write tests. Your job is to work with the team to complete the sprint goals. That means communication and collaboration, which should happen very early in the sprint. You can begin working on test designs and test plans well before the code is ready to be tested.

Bryan Oakley
  • 25,192
  • 5
  • 64
  • 89
0

You can do either, but typically you want to target regression testing with automation tests. That would mean I would do manual until your sure it's solid enough to be a regression test. This may be in the middle of a sprint for some functionality and may be in a future sprint for other functionality.

Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
mutt
  • 276
  • 3
  • 4
0

If you are going to test automatically you might as well create the tests upfront. This will help you define what behavior you are expecting and stimulates you to think like a client, it should make your software more usable in the end. And you will benefit from the test immediately as you are implementing the functionality.

Martin Maat
  • 18,218
  • 3
  • 30
  • 57
  • 1
    That does not work with UI testing tools like Selenium. You need a working and **stable** UI to be able to create the tests. – Doc Brown May 01 '17 at 09:00
  • @DocBrown: I don't think that's necessarily true. If you give me a specification for a new web page, I can begin (and maybe finish!) writing automated tests before the page is written. You simply need to collaborate with the developer so that you both agree on what the page structure is, what the element ids are, and so on. – Bryan Oakley May 01 '17 at 12:25
0

As was stated in another answer, when testing occurs should be a part of the definition of done. However, I do disagree with some of that answer, so I wanted to expand with experiences that I've encountered.

In a truly Agile environment, everyone is able to do everything. There wouldn't be someone 100% dedicated to testing, you would also develop skills to help with some basic UI work, or something else. However, we rarely ever live in an ideal world.

What I would recommend would be to do a bit of a hybrid approach. For your Definition of Done, I would say manual testing should be part of the Sprint the work is coded in. You know it works, and any bugs can be immediately reported, possibly fixed, before the Sprint is over so you can plan the next one. By focusing on manual testing, you become familiar with what the code is supposed to do. In the beginning of the next Sprint when you might not have as much to do, you can set up your automated tests that can run as part of the build process to prevent regression errors.

krillgar
  • 144
  • 10
  • I have never seen a sprint where there wasn't already plenty to do on the current sprint goals on day one. To write automated tests requires collaboration and planning, and that should begin in the first hour of the first day of the sprint. – Bryan Oakley May 01 '17 at 12:30