I’ve got a litany of test automation pet peeves that I need to get off my chest. From the absurd expectations of those wanting to implement automation to the so-called test automation expert who thinks it will solve all their problems if someone hands them the proper framework to copy. Test automation is becoming an IT cesspool. Big, important, life-impacting problems are being solved every day, yet test automation seems to bring out the worst in everyone!
If this article saves you explaining to an IT manager why they can’t automate 100% of their SAP regression testing in three weeks (or ever) or the time to read one CV with the same bullet points copied across ten pages, it will be worth it. Hopefully, somewhere between the lines of the forthcoming acidic rant, there will be some educational tidbits.
About the author
Lee Barnes has more than 25 years of experience working as a software quality and testing professional. So far, he has delivered test automation and performance testing ideas to over 500 companies. Lee sees his mission in closing the loop between development processes and testing and providing software testing services at a much larger scale.
1. Managing Expectations
In his article entitled “Test Automation Snake Oil”, James Bach calls out some common but very reckless assumptions about test automation. I’ve encountered one more of these in almost every project I’ve been a part of:
- Testing is a “sequence of actions.”
- Testing means repeating the same actions over and over.
- We can automate testing actions.
- An automated test is faster because it needs no human intervention.
- Automation reduces human error.
- We can quantify the costs and benefits of manual vs. automated testing
- Automation will lead to “significant labor cost savings.”
- Automation will not harm the test project.
You may think that the article was written long ago and wonder if any of this is still relevant? Yes, I know it was written in 1999 but it could have been written yesterday. The article is about principles, not practices and the principles are the same today as 20+ years ago. If this all sounds foreign to you, don’t worry. It’s understandable given all the marketing nonsense we’ve been subjected to over the years. However, if you’re fundamentally against what James is saying, you probably should not ask me to help you with test automation.
Now, back to the rant. One of my biggest pet peeves is unrealistic expectations that flow from the reckless assumptions pointed out by James in his article. I’ve lost count of the organizations that think they have implemented test automation just because they’ve installed a tool. Was your system magically completed when you implemented Azure DevOps? Of course not, and it’s just as ridiculous to believe that this is the case for test automation.
While we’re on the subject of unrealistic expectations, let’s talk about the people that organizations ask to implement successful test automation. Just because you have the most skilled testers, business analysts, etc., does not mean you have a group of budding automation engineers. A vendor might insist their tool is “codeless” and can be used effectively by anyone who can operate a DVR but the reality is that the skills required to be successful with automation are pretty specific.