6

From Wikipedia:

Quality Assurance refers to administrative and procedural activities implemented in a quality system so that requirements and goals for a product, service or activity will be fulfilled.[1] It is the systematic measurement, comparison with a standard, monitoring of processes and an associated feedback loop that confers error prevention.[2] This can be contrasted with quality control, which is focused on process output.

You can read the following article to understand the difference between QA and testing (quality control): Link

What I'd like to understand is whether there is a place for QA engineer who is not a tester in Agile/Scrum. Agile is deliberately short on processes and documentation (its even in the manifesto), and that's what QA specialize in. So should there even be a QA engineer in Agile?

I myself am not a QA engineer (I am a test lead), but I have a friend who is, so that's why I am asking.

Glorfindel
  • 3,137
  • 6
  • 25
  • 33
Eugene
  • 1,943
  • 1
  • 21
  • 35
  • Are you interested in a particular type of organization, especially one in a rigorous environment (medicine or healthcare, aerospace, automotive), perhaps that is adherent to something like ISO9001 or something similar? – Thomas Owens Mar 16 '14 at 20:05
  • We work in FDA regulated company, and my colleague, a QA engineer, wants to do quality assurance in addition to what is required by FDA. My recommendation would be against that. I don't see how process control is useful in an agile environment (that is besides what is mandated by FDA) – Eugene Mar 16 '14 at 20:08
  • 1
    I'm wondering if [this question and its answers](http://programmers.stackexchange.com/questions/81427/can-agile-and-iso-9001-interact-well) may be useful. I'm not that familiar with the FDA - I have experience with working with agile development within the context of CMMI and AS9100 (which is an aerospace version of ISO9001). If it's a CMMI-rated organization, the SEI has publications on combining CMMI with the agile methods as well. Could you provide more information about what the FDA mandates and what additional process monitoring and control is being proposed? – Thomas Owens Mar 16 '14 at 20:11
  • Going above and beyond what is mandated by higher levels - customers, regulatory bodies, corporate mandates - is not agile. It's definitely not lean, either. That's not to say that QA and process improvement doesn't have a place in agile organizations, it should be done in a way that doesn't add delays in the process or bureaucracy or reduce the power of the team (meaning the development team) to make changes in how they work. – Thomas Owens Mar 16 '14 at 20:15
  • What she would like to do for example is for us to create a bug life cycle and then make sure everyone adheres to it. The issue is that some of our developers use Excel, some write mails, some use TFS. And its all pretty fluid. – Eugene Mar 16 '14 at 20:19
  • 1
    I think I have enough to write an answer now. Let me take a stab at it. – Thomas Owens Mar 16 '14 at 20:20
  • She also wants to have a process for software design, a process for user experience design, and such. The issue here is that we change our processes all the time, and not everyone follows the same processes – Eugene Mar 16 '14 at 20:31

3 Answers3

10

I think that there is indeed a role for quality assurance in an agile organization, especially one that has to adhere to a quality management system such as ISO9001, ISO13485 for the medical device industry or AS9000 in the aerospace industry, for the purposes of meeting customer requirements, regulations, or corporate mandates.

There are two important things to keep in mind. First, quality assurance is not just interested in the quality of the final product, but all intermediate work products. You can view software development as a black box that takes in some form of customer requirements and outputs software (and perhaps support for that software). However, during that process, the development team does things - transformations of those requirements into something usable to design and build software, creation of designs, prototypes and simulations, production code, test code, supporting scripts, manual procedures (for testing, deployment...), user documentation, and more - any of which can have defects which would impact downstream activities that rely on that information. Quality assurance is concerned with not only how defects are found in these work products, but how those defects are prevented. Second, quality assurance is also interested in process quality - how well we are doing the work. Although the obvious thing here is the quality of the work products, it's also how long it takes to do the work (among other things).

Two of the tenets of Agile are to favor "individuals and interactions over processes and tools" and to favor "working software over comprehensive documentation". However, neither of these say that you shouldn't have a process or that you don't need any kind of documentation. If you're trying to implement a quality management system, it's likely that you are required to have some level of documentation that can be compared to the work that is actually done during auditing. I think that these Agile ideals are better expressed in a few of the concepts of lean software development - eliminate waste by removing delays in the process, reducing bureaucracy, and improving internal communication, amplify learning, and empowering the team to make decisions about how to best achieve the goals.

In addition to being responsible for internal auditing, a QA organization can also help support agility or lean software development by being responsible for helping to detect waste, find delays, and ensure that what is documented gives each development team sufficient leeway to implement process requirements (levied by customers, regulatory agencies, corporate mandates, etc.) in the best way for the individual project but still having some level of consistency across the organization to allow for people to transition between projects without having to relearn an entire new set of processes and methods on top of the technical learning curve.

However, based on some of your comments, it seems like the direction that this is going in is not Agile or lean. It's a fine line to walk on how detailed to make your process documentation. If it's too vague and gives each project team room to implement anything that they wanted to, leading to wide variations in process implementations (and therefore process quality and likely product quality) across the organization. If it's too specific and detailed, it's likely to add bureaucracy and comprehensive documentation would get in the way of interactions and working software. There needs to be balance between agility and the ability to obtain information about process quality (through measurements and some level of consistency).


See also:

Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
  • 1
    Good answer, good digging for what the OP was actually after. – Dave Hillier Mar 16 '14 at 22:17
  • Thank you for the thoughtful answer. What I would like to know is how come non regulated agile companies don't have QA people? The vast majority of people on this forum haven't even heard of such role. – Eugene Mar 17 '14 at 05:01
  • @Eugene I'm not really sure. Perhaps it's the same reason why few companies outside of these industries look for people with continuous improvement or software engineering process experience, outside of management roles. – Thomas Owens Mar 17 '14 at 10:39
  • 2
    "You can view software development as a black box that takes in some form of customer requirements and outputs software". Actually, agile software development can't be viewed that way at all. Customers must be involved in software development and requirements are changed/detailed/rethought *inside the black box*. That's why we use the word *agile* at all. A development process is *not* agile when it follows all the ceremony of an agile methodology, but there is no actual change in scope within the black box. It seems to me that it marks quite a change in defining "quality". – Sklivvz Mar 26 '14 at 12:09
  • @Sklivvz It's still a true statement in agile methods. In agile methods, the customer representative is a participant in the processes that occur inside the black box of software development and is not an outsider to the process. Even in more plan-driven methodologies, there can be some customer visibility. For example, in his definition of waterfall, Winston Royce called for external design reviews and acceptance reviews, and many organizations actually do have external requirements, preliminary and detail design, and acceptance test reviews with external stakeholders. – Thomas Owens Mar 26 '14 at 12:17
5

I'm not sure that I understand the difference (or how it is relevant) between "testing" and "quality assurance". In the end to do QA you must do testing.

QA, as you define it, sounds a bit against the Agile manifesto's value of Working software over comprehensive documentation. Do you really need all that process documenting? Are you trying to squeeze an existing process into an Agile one. If you really need the documentation you should add it to your definition of done.

Agile methodologies prefer "generalizing specialists" who do vertical slices. This means any team member should be able to complete any story. Developers should test, QA should test and everyone should work to build quality in.

Agile teams use retrospectives as the point where they improve their process.

In teams that I've worked in everyone is responsible for quality. Typically there is a lag between something testable in an iteration/sprint being ready for a tester. With this time, you can get your QA activities done just in time for testing of stories. Be careful for this introducing scope creep within an iteration.

Many Agile processes are not prescriptive about that level of detail and it is left to the team to decide. Would a self organising team ask for a QA specialist?

Dave Hillier
  • 3,940
  • 1
  • 25
  • 37
  • 3
    In an organization adhering to something like ISO9001 or AS9100, yes, having a documented process is important for auditing by external organizations, such as customers or governing bodies. A QA organization is not just concerned with product quality, but also process quality and the relationship between them. A QA organization would ensure that documented processes match reality (and either the documentation or reality is changed, as appropriate) and that the processes in place are appropriate for finding and containing defects early in the development lifecycle. – Thomas Owens Mar 16 '14 at 20:00
  • 1
    That's what I was thinking. QA is useful only when you are audited by some government agency. Otherwise it seems like an overhead to me. – Eugene Mar 16 '14 at 20:03
  • @ThomasOwens fair enough, I'm not that familiar with documentation heavy processes. Mike Cohn's book Succeeding with Agile has a description of how to combine CMMI & ISO9001 with Agile. It's quite thin on details... – Dave Hillier Mar 16 '14 at 20:04
  • 2
    They don't necessarily have to be documentation heavy, [as evidenced by this question, which includes an answer from a former ISO 9001 auditor](http://programmers.stackexchange.com/questions/81427/can-agile-and-iso-9001-interact-well). It's just a matter of being able to align how you do work consistently across the organization by pointing at something that's controlled and monitored. – Thomas Owens Mar 16 '14 at 20:08
  • At the extreme - difference between QA and testing. QA means you can manufacture single use devices that people can rely on. Testing means every single use device you manufacture goes in the bin. Software typically thinks of QA===Test because nothing we manufacture is one time use. – mattnz Mar 17 '14 at 02:04
  • 1
    Audits are useful regardless of whether there is a legal requirement to have them. 'Audit' is simply another way of saying you're checking that people are doing what they say they are doing. Whilst trust is important, of course it is, any reasonable investment has a requirement for checking authenticity. All business is a form of investment, all audits are a check of authenticity, QA is an audit of your quality processes, and all development processes that are worth using should pass, agile or not. – Rob Baillie Mar 17 '14 at 14:03
2

If you read the publications of the founders of the quality movement (Shewart, Demming, Juran) you will quickly come to the realization that quality is an economic issue. Lack of quality means there is a lot of waste. Quality improvement is the process of driving waste out of the process. Shewart, Demming and Juran were dealing with manufacturing but the same idea applies to software development. In software development the primary source of waste is rework (fixing errors, retesting, redeploying) always assuming that the customers functional requirements were met. The next most expensive problem is not producing something the customer wants to buy. Software quality also involves the user. Poor design or lack of reliability can increase the cost of using the software.

The role of Quality Assurance is not to test the product although test data is of use if it is recorded properly. The role of quality assurance is to investigate possible ways of reducing the number of problems introduced and reducing the cost of correcting them. This is not limited to code defects.

One of the worst processes for conduction any engineering development is to make designers/developers responsible for features. The problems arise when more than one developer or team has to modify the same component to provide the desired functionality. The result is defects that are not discovered until after integration (i.e. in system level testing). In the ideal situation, system testing should not find any defects at all. If a significant number are found, it means that there are a huge number of defects remaining in the product even after all the testing is done. This results in an unreliable product.

The answer to this problem is component or subsystem based development in which only one developer or team may modify any given component of subsystem. In a feature oriented shop, the role of the QA people would be to help the teams transition to a subsystem oriented process.

In any other undertaking, metrics are collected and used to predict the future. In many software shops the key metric is either defects found or defects open. But this is a useless metric for improvement. The key metric is the number of defects still in the work products. Requirements defects (assuming there are requirements) end up in the design along with the errors made producing the design. Design defects (assuming there is a design) end up in the code along with the coding errors. Quality Assurance should help to find ways of minimizing the number of defects carried over from previous work products by using the metrics collected when these items are produced and verified.

Quality Assurance is not responsible for the quality of the products. The people who produce them are responsible. Quality Assurance is responsible for discovering the messes made along the way and pointing them out to management so they can be addressed before they become serious and costly issues.

  • But how does this work in an Agile environment? In your second-to-last paragraph, you mention that metrics can be collected, but what metrics and how do you collect them? – Jay Elston Jan 06 '16 at 15:38