I'm a student working on my B.E(CS) and my question is the following:
My question: Is testing of software needed?
Yes. Because no matter how good you are, you can't think of everything.
For the same reason that a chef tastes his food while cooking it.
I work with someone that thinks like this, he thinks that because he is a senior programmer he no longer needs to test his code. The company does not understand how dangerous this attitude is and instead of firing him outright they hired more programmers to tackle the bug backlog. Not knowing where this backlog comes from they think it's part of what programming is all about. Basically we have 3 programmers that work under this mode and a team of 20 that do nothing else than test for and fix the bugs these three create.
ABSENCE OF PROPER TESTING KILLS.
So unless you are GOD or whatever version of some perfect all knowing being (now this I'd like to see) or unless you actively want to get fired very fast I strongly suggest you start testing.
Software is written by people.
People are imperfect and make mistakes.
As the complexity of an undertaking goes up, the potential number (and impact) of mistakes, oversights, or things forgotten goes up - usually exponentially.
So yes, testing is needed. It brings balance and perspective.
Will you get on a flight which runs an OS you know you used on your laptop and gave you a screen of death in your favorite color? Think about it.
No coder is perfect. Far, far, far from it really. You need testing, and testers often bring in perspective (also known as use cases) which the developers were missing.
Do a search on the most famous software bugs on Google to know what I mean.
And at the college level, get some read on test-driven development, unit testing and agile practices to know where things are right now.
Testing is an absolute must for any non trivial (in size, or function) application that is to be actually used. You won't find a single developer who cares about his/her craft (as evidenced by their visiting of this site) that will respond and say that testing is not necessary.
In addition to what's already been posted, having a full suite of automated unit tests on any given application will make you more confident in future code changes. This higher confidence (as unit tests provide a BIG safety net) will result in faster code changes to existing applications (due to less backtracking/double checking)
Errare humanum est
There is no such thing as a bug-free software.
The most skilled developer writes code with bugs. Even if a perfect developer existed, there still would be bugs due to discrepancies between:
The perfect developer is only a part of the whole thing. And perfect developers don't exist.
Most of real-life programs:
a) contain hundreds lines of code or more, scatterred across numerous files;
b) are developed by more than one programmer;
c) used in environments that are different from developer's environment
Thus if you won't check how program works in real-life situation, the chance that it won't work at all would be close to 100%.
In addition to all the other great answers, even if you know its perfect and bug free, think about the other programmers who have to deal with your code in the future. They won't know it like you do and will want to rely on your tests to ensure they haven't broken anything after they've made a change. This of course also applies to yourself after you haven't seen your code in a year's time!
YES.
Here's another slightly more subtle perspective that hasn't quite been covered yet:
Never underestimate the need for "independent verification". It's the same reason why it's good to have a few independent editors go over your work before submitting a large piece of writing for publishing. No matter how good a writer you are, you will occasionally brainfart - and write something like "in" in place of "it", or something. If you re-read it yourself, even quite carefully, you'll still usually miss it, because your brain automatically accepts your thought process flow as correct, and glosses over the error. To a fresh set of eyes, that sort of mistake is usually quite glaring.
You get the same thing in programming: it's quite easy to get into a flow where either your code, or your basic "development testing" of your own code - looks correct because you're testing it and using it a certain way. But then when another pair of hands comes along and clicks things in a slightly different way or order, everything comes crashing down.
Now, of course, you could in theory go down the route of formally verifying every single possibility and logic branch in your code yourself, but for non-trivial software this will be far more expensive and time-consuming than having someone else bang on the code for you. And you'll still probably miss things that you never thought of.
What's not yet been touched: even if your code is perfect, you're still not safe. Compilers have bugs that can cause even perfect code to behave incorrectly after compilation. Operating systems have bugs that can cause a perfect binary to behave incorrectly when run. Hardware has bugs that can cause problems.
That's also why testing on one machine isn't enough for commercial products. They need to be tested under as many possible combinations of hardware and software they can encounter in the wild as is feasible.
The leader of the team writing the software for the space shuttle flew out before every launch to sign that the software would not harm the shuttle.
What would you think gave him the confidence to do so?
Smells like a homework question.
Is testing in the software field needed?
Yes. Absolutely. At all levels. Outside of a few specialized domains, we're not yet at a stage where we can mathematically prove our code is correct against specific bugs (at least not in a reasonable time frame), so we have to throw rocks at it to see if and where it breaks.
If we create a software with great care, then why should we test?
Testing isn't just about finding coding errors. It's also about making sure that you've met all your requirements, and that the overall system performs as expected. If I have a requirement that a failed transaction must return a specific error code, then I need to write a test to verify both that the functionality exists and that it works correctly.
And all that assumes that the specification and the design are complete, correct, and internally consistent, which is often not the case. Even if you meet the specification to the letter and follow the design down to the last dot and semicolon, if the spec or design is bad, then there are going to be problems at integration time. Often, system or integration testing is when you find out that the specification itself is buggy and needs to be revised (see war story below).
After testing can we be sure that we have achieved this goal (the product/software works as intended) because we have done the testing for it? Is it possible?
No, not to 100%. We can't test every conceivable combination of inputs or paths of execution in any but the simplest code. We can't account for all environmental factors. We can't imagine all possible failure modes.
We can test to a point where we're reasonably sure there aren't any big problems. Again, this is why we need to test at all levels. Write a suite of tests to make sure your code handles edge conditions properly (bad input, unexpected results, exceptions, etc.). Unit test to verify that your code meets its requirements. System test to verify end-to-end processing. Integration test to verify that all components speak to each other correctly. Do usability testing to make sure that the whole thing works in such a way that customers don't want to shoot you.
Real-world scenario - I was working on a back-end system that occasionally sent updates to a GUI service for display in a table on the screen. During the project, a requirement was added to add filtering to the display (e.g., the operator could choose do display a subset of the entries in the table). Design mistake #1 - filtering should have been done by the GUI service (I have this quaint, antiquarian notion that display management functions should be the responsibility of the display management software), but due to politics and my inability to recognize problems before they become problems, that requirement was placed on the back-end service. Well, okay, no problem, I can do that. Filter state changes, I get a message, and then I send a create or delete message for each row in the table, because that's how the interface works (design mistake #2 - there's no way to send updates to multiple rows in a single message; we couldn't even send a single "clear" or "delete" message to clear the entire table).
Well, everthing works fine during development; unit, system, and integration testing show that I send the right information and handle the filter changes correctly. Then we get to usability testing, and the whole thing falls down hard, because the volume of data was overwhelming. The network latency between my backend service and the GUI was on the order of .15 to .25 seconds. Not bad if you only have to send updates for a dozen rows or so. Deadly when you have to send updates for several hundred. We started getting bug reports that the GUI was freezing up after changing the filter state; well, no, what was happening was that it was taking on the order of several minutes to update the display because the bone-headed update-one-row-at-a-time message protocol couldn't handle a real-world scenario.
Note that all of that could have and should have been anticipated by everyone from the prime contractor all the way down to little old me if we had bothered to do even the most basic analysis beforehand. The only defense I'll offer is that we were closing out the second year of a six-month project that was going to be junked almost immediately after delivery, and we were all just desperate to see the back of it.
Which brings us to the final reason to test - CYA. Real-world projects fail for a variety of reasons, many of them political, and not everyone acts in good faith when things go wrong. Fingers get pointed, accusations get made, and at the end of the day you need to be able to point to a record showing that at least your stuff worked as it was supposed to.
You are constantly testing code just by compiling and using it. In some IDE you are getting sanity checks as you type. Unless you never actually run your code you are doing testing.
How much you test is really the root of this sort of question and the answer to that comes down to risk. You test as much as it makes sense to test from a risk management point of view. Testing everything or nothing is usually impossible. Testing next to nothing is usually a bad move. Everything in between is fair game depending on the risk level and exposure of your deliverable.
All programs have bugs, at least to start with.
There have been some studies which converge on about 1 bug per every five lines of untested code.
A history lesson:
Back in the 1960s IBM needed a "NOP" program so they could execute some functionality in JCL without actually running a program. The developers came up with a one line assembler program in which the entire code was contained within its name "IEFBR14" the actual code being:
BR 14 * brach to return address in register 14
During its long liftime this one line program has been subject to 2 bug reports and five amendments.
Testing will always be done and bugs will always be found. Its just that either the testing will be done internally by your team, or the end-user will be the tester. The cost of a bug found by the end-user is tremendously higher than if it had been found by during testing.
I would suggest taking a good Fault-Tolerant Computing course. Careful design of software is just one of the pillars to achieving robustness in your software product. The other two pillars are sufficient testing and redundant design. The basic intent is to accommodate an exponential number of unknown failure conditions and to prioritize dealing with some of the known ones:
1.) eliminate as many possible failures through design and proper implementation 2.) eliminate unanticipated failures from design phase and incorrect implementation through various forms of testing (unit, integration, random) 3.) deal with any left-over failures through redundancy (temporal => recompute, retry or spacial => keep copies, parity)
If you eliminate the testing phase, you leave yourself with only design and redundancy phases for addressing failures.
Also, from a product standpoint, your stakeholders (e.g. management, users, investors) will want some kind of reassurance that your product meets certain quality and safety specifications, criteria, etc. Aside from all this, haven't you tested the software that you constructed just to have a 'sanity check'? All these reasons make software testing more compelling.
Code refactoring is really faster when you have unit tests. A unit test also show you simple use of concrete function, so you have a little "howto" which can be really useful in big projects where programmers dont know exactly the whole code.
When you are Developing with TDD (test driven development) you dont have unnecessary getter/setters etc. You just create what you need.
To answer #3 of your question:
Having been a programmer and a software tester, yes, you can be sure you're meeting the requirements of the software during testing.
{putting QA hat on}
How? You can do this by tracing your tests from test code to acceptance criteria, acceptance criteria to features, and features to requirements. If you trace every single test up the design chain and they map to one or more requirements, you can be sure that your tests are being used to ensure code meets its requirements (although this brings up the notion of adequate test coverage, which is a another topic). If you can't trace a test up the design chain, then you're likely testing for things that aren't required, and this is a waste of time. Acceptance criteria can also include checking for undesired behaviors - you can test for those too, which gets you one step closer to a quality release.
{QA hat off}
No code is ever bug-free, just less costly over time when more effort is spent assessing its quality during development.
I have been a Software tester for 3 years now. Initially I myself was sceptical about the need to test, since I thought if the development department and project management do their job then no mistakes in software should arise.
BUT this is not the case.++++++++
Mistakes happen often, some of them critical to the operation of a project. There is also cross-browser testing( which mean testing on different existing browsers like SAFARI, FIREFOX, CHROME and Internet Explorer) and I worked on the project where simple interface buttons like YES and NO on a survey window where not working in all browsers only some of them.
I worked on internet page testing, and was testing simple TEXT CHANGES and thought to myself that no way on earth there could be defects on this simple work, but no it does happen.
Also I have seen when new developers join the team and are given an update work piece to an existing complex internet application form with a number of linkages/calls to external pages, mistake took place because of lack of the communication between old and new developers, for various reasons ( no time to educate, no will to educate and so on).
YES
Imagine your software is only a logical function AND(b1, b2) where b1 and b2 are only bits. With that you need 4 test cases to be sure that your software is error free, if your surrounding environment delivers exactly what they were specified for.
Now your system is made by a lot of functions whose simplest one is way more complicated than that logical function. How would you make sure it is error free ?
(to be continued)
