Common mistakes in software testing and how to overcome: "There are common mistakes we do in testing. I am giving few examples from it.
1. Regression testing: we never follow
2. Developers word: this is my code; 100% accurate
3. Always testing with only the positive scenario
4. While reporting defect never thinking of developer’s situation or mind
5. Never explore the application
6. Missing code in the final deployment
7. Performance never observed or too late
8. Testing time estimation
9. Start testing too late
10. Clients always like to change the requirements
11. Test is continuing still code changes in the final deployment; not a full cycle testing/complete testing
12. Recruiting testers from the ranks of failed programmers
13. Using testing as a transitional job for new programmers and anyone can test
14. Better understanding and efficient communication between programmers and test engineers
15. Programmers cannot test their own code, not happy with Unit testing and testing script
16. Liking more execution of test rather than design/ un-reviewed test designs
17. Not but testing also need to take as group work not isolated than development
18. UI issue or cosmetic issue never gets priority by developers
19. Attempting to automate all tests/ expecting to rerun all manual tests
20. Test coverage
1. Regression testing: Whenever its time to regression tests; we never have enough support for time, talents and test environment. If we like to regression test properly then help of automation script base test required. But initial cost and time of automation is more and expertise for automation test is not available. So, never come up with better solution for regression test and this is basically partial test and obviously it’s dangerous for live project like www.ourleague.com
To overcome this issue we need to allocate enough time, talent and test environment.
2. Developers word: this is my code; 100% accurate: Over confidence is dangerous same as lack of confidence. Sometime developer would say: this is my code; 100% accurate and never believe.
I know how super smart programmer you are and obviously I will appreciate this but I will do my testing that should be fulfilled my already planned test. If you believe this type of word simply you will get killed.
3. Always testing with only the positive scenario: Some or for many reason we do only positive testing and basically this is not testing. Software will work, everyone knows but why we are testing? We need to break the box so that client never gets the panic.
The programmer also tests their own code and they work for running the application. So, if we test engineer do the same task with more exploring the positive scenario that’s not sounds good. We need to cover negative scenario as well.
Here also time, cost and test environment is the de facto.
4. While reporting defect never thinking of developer’s situation or mind: This is a very common mistake for test engineers, while reporting defect they never think from developer mind. We always report defects and think it’s understandable. This is something like handwriting; if someone thinks that I can read my hand writing and saying: other people can read because I can read it.
Defect reporting is very special quality for a test engineer. Before reporting please reproduce it properly and try to make the path of replication clear and concise, give idea to the developer what makes this defect. Then only they will respect and will give us value for saving their time by proper defect reporting. Every time we must think it’s not me but others also need to be understandable. Write it with text, if needed put images if more then make video file.
5. Never explore the application: Exploring the application is very important so that we can rarely miss any defects. Most of us never try to explore the application; only trying with limited scenario. Later clients find bugs and the test engineer cried loudly!!! What makes me miss this silly mistake; it’s needed exploring.
For example, MS Word; we can print several ways 1. File option 2. Ctrl+P 3. Print button 4. Preview and Print button and so on
If there are several input fields then changing the sequence may work.
How to do this I cannot give any answer but just one word “EXPLORE”
6. Missing code in the final deployment: This is the worst case ever we worked. This can messed up every good works, one of our build is ready for the live suddenly we found mixing of old and new codes. Unfortunately it’s not the entire version code but partial code and finding and merging it more difficult and the final moment of deployment.
I guess we need more awareness of code management tools and use. Moreover tag for the version is essential and every developer must be careful of checking in and out and if there are branches then the merging also important. We should take care of these before the final deployment and for this we can use checklist.
7. Performance never observed or too late: Application goes in live without measuring the performance that really crucial. How many users will hit at time? Or how will be the hardware configuration? These things need to be resolved before deploying in the production. Maximum cases we never care about the performance and if we think then too it’s late just before the deployment we used do some formal performance testing not enough for the live. As a result when server goes down; nothing could be done except cutting off the engineers sleeping time and heavy load work and returning home late night with panic.
We can avoid this panic if early performance tests plan and execution possible.
8. Testing time estimation: This is an issue always fussing between testing and development team. The programmer thinks about testing that it should not take longer time. Obviously maximum cases deployment get ready and test engineer starts test execution and it will be the eleventh hour for the shipment and not getting enough time to test. There are some tools to estimate the testing time but you know non can say the estimation is accurate; so many variables and dependency.
As a software test manager better is to define the task could be finish by timeline. If management asks about testing time estimation we could say by this time we could cover these area and if you give us time then we can cover full regression etc.
9. Start testing too late: It is another common problem with testing. Test engineer never get involve from the very beginning of the project. So, there were huge gap with test engineer and the project; may be the test engineer is busy with testing another project.
But you know it’s never been wise decision always it will kill our time. If the test engineer could be engaged with the project he/she could give better output. So, eleventh hour testing and humping and jumping won’t work to build better software. Continuous testing is always better than the traditional testing, it saves time and cost.
10. Clients always like to change the requirements: This happens for maximum clients and we cannot ignore it. There are lots of funny story about client requirement changes. Things like their requirement initially and finally far difference it’s because of unable to fix their requirements.
To cope up with the requirements we need very close interaction with the clients from the start to end of the projects. We have to understand the clients’ knowledge on the domain of the software and regular feedback from customer. It’s better to build the software sprint wise and feedback will be accommodated within the sprint to save panic and headache.
11. Test is continuing still code changes in the final deployment: Again common scenario; we have started our regression testing and supposed to be complete the regression cycle without having any more changes. But we CANNOT stop it, management or may be influence of present needs there will be code changes and deployment while we are half or some portion of regression testing.
Finally half done situation we will continue with regression testing with the code changes and deployment and you believe me this is not the full cycle REGRESSION test.
12. Recruiting testers from the ranks of failed programmers: This is simply horrible; some people got the idea TESTING is the easiest job in the world anyone can do it. I have faced this problem from my experience; management insist to hire those are failed in programming skill. Even I have seen one of the engineers in software testing even does not have the idea EXCEL needs to save to retrieve data for the future.
There is only one way to stop this type of situation that is showing how critical testing JOB is and how much important to make the software better to compete in the market to grab clients and users.
13. Using testing as a transitional job for new programmers and anyone can test: Here is another thinking of software testing. Management why to unutilize our new programmers they can do testing job and that is enough. Even some of them are thinking anyone can do testing…
14. Better understanding and efficient communication between programmers and test engineers: This is a vital point to build better software. It’s a TEAM work not one man show job. Because one man can do many things but his concentration will be focused on some area and other area will be dark enough to spoil it. When there are better communication channel with programmers and testers its make huge different than NOT.
Regular team meeting and milestone of the project and understanding between team members are essential. This type of gap can mess up in the end. We can use tools for better communication and regular team meeting achieve this goal.
15. Programmers cannot test their own code, not happy with Unit testing and testing script: I have seen so many programmers; they are great in their coding skill nicely implementing critical logic but when you say you have to write test code. That’s make over-burden to them but I guess it should not be like this. Anyone who is passionate on developing software should not afraid of test code. I know it’s again headache to write the codes but later it will bring the best fruit. And may be always its better to test code with peer or something like that to bring better results e.g. we review our test script/test cases.
16. Liking more execution of test rather than design/ un-reviewed test designs: This is major part of testers and maximum we failed here to make it. Whenever we develop test plan and test cases/scripts once execution starts we never update or re-design the test plan and test cases/scripts. Everyone wish to execute/testing more rather updating or re-designing the test plan and test cases/scripts. As a result we unable uncover critical issue in the end.
17. Not but testing also need to take as group work not isolated than development: Some people have the idea that testing is a different kind of job and so this talents are isolated from the TEAM. We need to remember again and again it’s a TEAM work. This lead to miscommunication, misunderstanding and the other type of mis…
18. UI issue or cosmetic issue never gets priority by developers: Huh! This is very minor issue and we are not going to fix it- its development view. Developer used to blame testers you unable find bugs with the functionality and finding this type of low priority issues. When we met someone we glance each other face not the internal things e.g. how soft minded the person would be? That part will come later. Same way the UI issues might be not critical as functional breakage but again this is like the given example. So many clients are there; they liked to see the UI is accurate first otherwise they will not go for touch it.
19. Attempting to automate all tests/ expecting to rerun all manual tests: Automate test is superb; it’s faster, efficient, less time and cost consuming. True, but we cannot automate ALL the features. Nothing is above MANUAL testing. Some will understand that we will automate our testing process and it will take care of manual testing. That’s what there is certain criteria before going for automation tools and again this cannot fulfill the gap of manual testing. We should have better understanding of it.
20. Test coverage: How we will measure test coverage? We know there is NO way to complete testing. But we have stop our testing process and it cannot be an infinite process. There are certain points to set the test coverage. In my opinion, we can set the criteria depending on nature of the project and always it will differ one to another.
We can have a look here for complete testing and test coverage: http://www.kaner.com/pdfs/impossible.pdf
http://asusrl.eas.asu.edu/cse565/content/coverage/coverage.pdf
References:
1. Classic Testing Mistakes, Brian Marick, Testing Foundations
2. M. Cusumano and R. Selby, Microsoft Secrets, Free Press, 1995.
3. Michael Dyer, The Cleanroom Approach to Quality Software Development, Wiley, 1992.
4. M. Friedman and J. Voas, Software Assessment: Reliability, Safety, Testability, Wiley, 1995.
5. C. Kaner, J. Falk, and H.Q. Nguyen, Testing Computer Software (2/e), Van Nostrand Reinhold, 1993
6. Cem Kaner, J.D., Ph.D. (http://www.kaner.com)
7. http://asusrl.eas.asu.edu/srlab/
