7

Background:

Developer is the best person to know/understand the dark corners after any development/enhancement of enterprise software, compared to QA technician.

Developer can assess the depth/breadth of production problems that can arise from such dark corners.

FOR Quality reasons(only) but not for cost cutting, my new employer does not maintain separate QA team and shifts the responsibility to developer(as better owner) to write/append/automate/validate test cases. So, all new test cases are append to existing test suite, as part of process.

My employer, manufacture network switches with their customised Linux based OS on switch.

At my workplace, the first rule that is followed in regression testing is to avoid regression testing. We avoid regression testing by testing any new testcase written for every feature, on every platform, for every release. If the test fails, then matches the log with database of known failures and points out some fingerprints.

Question:

1) Does separate QA team add any value in a software development life cycle?

2) Does it add value for a Quality product delivery, if QA team is formed that has ethical hacking skills? For software products that run on network

overexchange
  • 2,245
  • 2
  • 17
  • 47
  • I agree with you that the "developer is the best person to know..." Whenever I turn an application over to QA, I've already thoroughly tested it myself. And I can usually tell from conversations with them that they're pretty clueless about the internals, e.g., at what point input causes the program to change from one algorithm to another, so that boundary region has to be carefully tested. They just apply their own standard techniques. But I just figure, "Everybody has to make a living." Nevertheless, I've occasionally been surprised, and they turn up a bug I failed to find. But not very often – John Forkosh Mar 17 '17 at 22:55
  • Developers are allowed to give advice on which areas may benefit from more testing or how best to test some functionality to QA in most cases. There's no reason the developer needs to hoard the knowledge of where the "dark corners" are. – Derek Elkins left SE Mar 17 '17 at 23:18
  • @JohnForkosh I agree with you. Most of the managers run QA team to save their own job, as they need to show tge number of employees reporting him.There are very less scenarios where u need separate QA team. – overexchange Mar 17 '17 at 23:22
  • @overexchange You make a change and all the tests pass. Great! Now, how do you know that the tests you wrote are testing for the right things? – Derek Elkins left SE Mar 17 '17 at 23:29
  • @DerekElkins I don't hide anything. But I've found some QA people just aren't interested (and by "some" I'd guesstimate ~2/3). They've got their own way of doing things, and that's what they're going to do. I just usually ask, "Is there anything I can tell you?", or something like that, and follow their lead. But they typically don't want to know much more than how to run the application. And if you try to tell them more than they want, they sometimes just get annoyed/snotty/whatever about it. And as a contractor, I usually try to do my best just to keep everybody as happy as I possibly can. – John Forkosh Mar 17 '17 at 23:39
  • @JohnForkosh That comment wasn't directed at you. Nevertheless, the purpose of providing this kind of information is to help focus QA effort. It isn't usually necessary to go into technical detail. It can be as simple as saying "I have less confidence in this part of the application". In theory, tests should be derived from requirements which are independent of the code, so QA *should* only take your advice as a suggestion and perhaps one to be dismissed upon occasion. On the other hand, I won't deny that there are a lot of bad QA engineers. – Derek Elkins left SE Mar 18 '17 at 00:14
  • @DerekElkins Oops, right, not directed at me. The other comment was, and I failed to explicitly distinguish, maybe because your remarks seemed relevant to what I'd said. Anyway, yeah, many bad ones. I've inferred from experience that those are usually dedicated QA "engineers" (and I use the word loosely:) who've never been developers themselves. Good ones are usually developers who've been given a QA assignment. Of course, that's just a generalization, but I'd guess it's more often true than not. – John Forkosh Mar 18 '17 at 00:20
  • @JohnForkosh I think you have a confusion of correlation and causation there (though there's certainly *some* causation). I've definitely worked with good QA engineers who've had no developer experience. A QA engineer who was formerly a developer is likely not at all cheap. My impression, though, is that many companies view QA as largely a "monkey" job and hire accordingly. – Derek Elkins left SE Mar 18 '17 at 00:37
  • Don't underestimate the value of someone pressing buttons in ways the developer never thought of before the product is shipped to the field. Even if that's all QA did, which they do a lot more, then they are worth it. As a developer, you'll probably make sure the code works as you expect but won't spend a lot of time figuring out how to improperly use the system. You may technically be right that it was operator error that caused the database to become corrupt. But in your customer's eyes it was your software that is broken if it let the operator do the wrong thing. – Dunk Mar 21 '17 at 21:10
  • Possible duplicate of [Should a developer also act as a tester?](https://softwareengineering.stackexchange.com/questions/102507/should-a-developer-also-act-as-a-tester) – gnat Jul 21 '17 at 09:19
  • see also: [Should QA be part of the development department?](https://softwareengineering.stackexchange.com/q/129091/31260) – gnat Jul 21 '17 at 09:20

4 Answers4

10

I would say that it depends on the system being developed and tested.

The developer might have the best understanding of the code, but that doesn't mean they fully understand all the business cases that the code is used for.

For example, consider a web application used by insurance brokers to manage and process policies and claims. There are many many rules and business cases that go into such a system. It's highly unlikely that any one developer will be able to know all of it in great depth. Heck, most business users probably won't know the whole system (although they will have a better business-level understanding of more of it than developers).

In this case, a separate QA team will have received special training on the business flows within the application. Chances are, they might not even be developers, they might be Business Analysts who have transitioned into QA due to their extensive experience. Or they might be developers who are more interested in the non-code-related facets of the system. They won't understand all of the technical details, and they won't need to. They will execute business cases that probably go beyond what the developers know or even need to know. Developer testing (usually done by developers locally on their own workstation, to ensure that their code works at more than just a basic level) and QA testing (done by QA teams to ensure larger sets of business cases are satisfied) are two distinct phases with different kinds of tests and expectations.

What value does this add? You have seasoned, experienced QA testers who have a very good high-level understanding of the system as a whole. They know how to test the system to ensure that all parts are properly covered, based on the changes made by the developer. And it frees up developer time from something that really is a full-time job and requires extensive business training. All of this can lead to fewer bugs making it to production and higher over-all quality, and happier customers (and that's the whole point, isn't it?).

In my experience, the larger/more complex the project is, the greater the value added by having a full QA team. On smaller systems with a single (or small handful of) developer and simpler use cases, the value added is not so great, and in some cases, having a separate QA team could be redundant (and costly) if the testing is simple enough for the developers to manage on their own.

FrustratedWithFormsDesigner
  • 46,105
  • 7
  • 126
  • 176
  • Developer don't need to know/remember complete test cases from scratch because it is already part of Test bed(like HP QC). What developer remember for that cycle is his own new test cases gets validated and does not fail already existing test cases. – overexchange Mar 17 '17 at 18:17
  • 1
    @overexchange: For unit tests, I agree with you. QA teams usually test at a level beyond unit/integration tests. If your QA team could be *easily* replaced with good unit & integration tests, then perhaps you *don't* need a QA team. – FrustratedWithFormsDesigner Mar 17 '17 at 18:25
  • Level beyond unit/integration tests? Can u help me understand more on this? – overexchange Mar 17 '17 at 18:43
  • @overexchange: Unit/integration tests are automated. QA testing involves a real human being working with the application to ensure that it does what it is supposed to do. The tester works with the UI, so they test and give feedback on it. For complex interfaces (again, consider the insurance application example), this usually proves quite valuable. Many business flows could involve 10-20 "screens"/pages - it's easy for a single developer to miss things in this situation, so having another person walk through it can catch bugs and problems. – FrustratedWithFormsDesigner Mar 17 '17 at 19:05
  • can I generalise and say, for GUI applications only, we need separate QA reams? – overexchange Mar 17 '17 at 19:07
  • @overexchange: QA testers can also execute business processes and workflows that are too complex or difficult to automate. If all of your application's process and workflows can be easily tested 100% via automation, then that's a great thing and you are very lucky! They understand the business at a much deeper level so they can see problems that might not have been anticipated in the original requirements, that a developer would not know about or understand. – FrustratedWithFormsDesigner Mar 17 '17 at 19:08
  • @overexchange: No, you cannot generalize that. A *simple* GUI application probably doesn't need a separate QA team (very simple example: a GUI calculator). And some CLI-based applications I've seen have embedded interactive prompts/flows and really *should* have separate business-expert QA teams. – FrustratedWithFormsDesigner Mar 17 '17 at 19:09
  • 4
    @FrustratedWithFormsDesigner To emphasize your point, I would add no amount of testing by the developer, automated or otherwise, will help if the developer has misunderstood the requirements. Even worse, if that developer then enshrines that misunderstanding into a test case. – Derek Elkins left SE Mar 17 '17 at 23:29
9

FrustratedWithFormsDesigner brings up one case where separate QA is useful - a complex system.

There is another case where separate QA may not only be useful, but mandatory - regulated environments. There is a notion of independent verification and validation. Independent means that it's done by a person or group of people who did not write the production code. In some environments (I have experience in aerospace and healthcare) and under some conditions, it is mandated that there is some level of independent verification and/or validation. In all of my experiences, the development team is responsible for some level of testing to ensure that garbage isn't turned over to the independent QA testers. In addition, the independent testing may be manual testing, automated testing, or a combination of both.

There's also the question as to what "separate QA" really means. Does it mean embedding QA staff onto each development team or does it mean having a handoff from a development team to a QA team. Independence can be achieved in both models.

Community
  • 1
Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
  • 1
    Ah, good point about regulated industries! +1 – FrustratedWithFormsDesigner Mar 17 '17 at 19:10
  • 1) No separate QA, I mean, no QA resource is merged in development team. Coder holds the responsibility that any QA team member would do. 2) Yes, car manufacturing company has to have longer test cycle than manufacturing cycle for a specific car to be tested, with all air dynamics. – overexchange Mar 17 '17 at 19:14
  • 1
    @overexchange My answer addresses both cases. In some industries and environments, you need a separate person to do QA other than the person doing development. This person can be on a separate team or embedded onto the development team (as long as they don't do development) - both are equally valid. – Thomas Owens Mar 17 '17 at 19:18
  • Instead of car/aerospace, if I take more relevant example, I regulate malloc() to always allocate at virtual memory address above 4 GB virtual address space, for a 32bit software that is ported to 64bit. But, developers are more relevant to test such things, because tester is reluctant to understand what virtual address is? And their boundaries in 32/64 bit arch – overexchange Mar 17 '17 at 19:26
  • 1
    @overexchange: and if the development of malloc was a regulated industry, you probably *would* need *independent* verification (by a QA team that is either within your organization but distinct from the dev team or more likely is external to your organization entirely) of some kind that you coded it correctly, and then once you have this verification, you would able to say "Our malloc passed regulatory verification and can be used for $x, $y, and $z, with use for $a' pending." – FrustratedWithFormsDesigner Mar 17 '17 at 19:29
  • ..the QA team doing the verification would need to be subject matter experts and in this case, that would probably mean they would need to be developers, but they would still be a separate QA team. – FrustratedWithFormsDesigner Mar 17 '17 at 19:30
  • 3
    @overexchange: Also, you seem to think that all testers don't understand programming concepts. Some of the best testers I've worked with had previously been developers who decided to work more on Business matters and testing (for whatever reason). *They* understood memory issues and they knew how to read a stack trace when one appeared on the screen. – FrustratedWithFormsDesigner Mar 17 '17 at 19:34
  • 3
    @overexchange You also seem to be thinking that developers do no testing if there is separate QA. That's false. Developers have a responsibility to turn over working code (as far as they understand working) to QA just as much as they do when they deploy code to production without a separate QA. – Thomas Owens Mar 17 '17 at 19:41
4

I definitely like the points discussed by @FrustratedWithFormsDesigner, but I think it is useful to consider the larger universe of testing types when it comes to a complete software system. I am sure there will be differences of opinion on who should do what -- and I welcome discussion :-) -- but the figure below is a quick sketch representing my experience and opinions.

As you will observer, there is a lot of overlap as to who should test what. Developers should be responsible for delivering to QA a complete and correct product (or feature) by means not just of unit testing, but some measure of the other categories of testing noted. Those that I have marked as "secondary" should be considered by developers, but not as their primary responsibility. That's why you need dedicated QA people, people with expertise in those areas.

The business stakeholders add another level of validation, and though their participation in testing will vary widely from company to company, it is, after all, they who deliver an acceptable or unacceptable product to the customer.

A couple key takeaways from my chart:

  • Unit testing is the sole domain of developers. QA folks with expertise in coding might want to peek at some of the internals but to help them with developing functional and other tests, not for unit testing.

  • QA is responsible for the whole system synergy, already discussed by @FrustratedWithFormsDesigner.

  • Security is everybody's business!

Testing Types by role

Michael Sorens
  • 1,346
  • 8
  • 11
1

Of course you have thoroughly tested yourself before handing over to QA. But the very fact that the developer is the most familiar with the application is the best reason to have someone ELSE test it.

The developer knows what the software is supposed to do, the way in which it is supposed to be used, etc., and it will be exceptional if he or she attempts to use it in the way a naïve user might, thus breaking it in unexpected ways. It will never be a black box to the one with the x-ray vision. Too often the developer is the the one the most removed from the real world use of all persons involved in the project.

MickeyfAgain_BeforeExitOfSO
  • 540
  • 3
  • 6