Continuous Delivery World: Is There a Place for Manual Testing?
June 13, 2017 - Quality Assurance and Testing
A while ago Agile methodology and some methods based on it (like XP, Scrum, Kanban etc) were considered the most effective approaches in software development process. A lot of companies continue to use them and as a result, develop high-quality products. Some of them even work with more ‘conservative’ models like Waterfall or Iterative, but to each their own, right?
Nowadays DevOps with Continuous Delivery are state-of-the-art and urge organizations to use Continuous Integration principles and practices.
Rapid progress and strong competition in IT sphere push experts, products, technologies, ideas, principles, and tools to be one step ahead all the time. Concerning software deployment continuous collaboration between Development and Operations ensures efficient progressive method that enables you to deploy a highly reliable version of the product at any time.
The primary benefit of using this method is the velocity of each phase of SDLC. To that end, it provides more time to market, improve the frequency of deployment, assure identification of defects on earlier phases of development and brings quality assurance.
Automation is essential for the implementation of CD approach. It’s used nigh in each phase of product development life cycle. Continuous Integration practice provides verification of each integration by an automated build and automated test. Together with Continuous Testing, they bring quick defect detection and rapid feedback.
When changes are pushed to the source code repository, the CI tool (like Jenkins, TeamCity, etc) complies code and runs unit tests. Then (if tests are passed) new build is deployed to the test environment and other automated tests are run. Usually, this process is implemented with ‘nightly deployments’. Every night new build is created and tested. When workers get to work they already have a clear picture of the actual state of the product – version of the build, the tests passed and failed, covered features and so on.
Just imagine if you had to do that all manually, how much time and effort would you need to do the same things every single day?
The thing is, automation is in the spotlight nowadays. It’s the best thing since sliced bread, especially as regards CD practices. However, some people can get the wrong end of the stick and think that manual testing plays second fiddle in software development. Even in the SDLC where everything is automated manual testing is still important. Moreover, there are some types of testing that machine cannot perform.
Exploratory and ad-hoc testing are decidedly manual activities. You could have a high level of test coverage and check a plenty of various scenarios automatically, but robots normally have none of such skills as observation, critical and analytical thinking. A machine does what you tell it to do. Having said that, you are not able to predict all your future ideas as to in which way to check a feature. It’s hardly possible to foresee and take into account all things you can notice in the process of manual running of tests.
Usability testing – something that is beyond the realm of possibility of automation. In terms of accuracy and counting the computer is at the top. A human being only can assess a product from a position of a customer. All functional tests could be passed, but from user’s perspective, some UI parts could be strongly uneasy. Manual QA engineers give an estimation of design, easiness and usability, etc.
Immediate testing of updated product is the preference of manual testing. Even small changes in UI can ruin your automation tests when a human being quickly adapts to the new interface. At any rate, time is needed to keep your automation test up to date and sometimes it’s hard to do it step by step with development. Furthermore, the more experienced an engineer is, the easier it will be for him/her to start testing new products and do it efficiency.
That fact leads us to the situation when automated tests fail and someone has to figure out the reason. If auto-test fails, it doesn’t immediately point to a defect. There are some other factors like connection issues, platform or architecture failures, minor interface changes and other things that can cause problems in the course of test automation. Every failed test case should be run again manually, just to make sure that everything runs smoothly.
It’s obvious that concerning Continuous Delivery methodology automation is of great benefit. And even if it seems that it is far ahead, bear in mind that there are both pros and cons of both automated and manual testing. One way or another, whatever you develop, there will always be a place for manual testing in software. And when you combine different practices, you will always get the best of both worlds.