Client:
A national cleaning and restoration services company
Industry:
Commercial and Residential Services
The Goal
Reduce regression test cycle time
A national cleaning and restoration services company struggled to maintain their field service management system. As they rolled out new functionality to their Salesforce Field Service application, regression testing activities became a bottleneck. Despite implementing test automation, they had little value to show for their efforts.
The Director of Quality Assurance approached Forte Group after attending a conference presentation by Chief Quality Officer, Lee Barnes. After hearing Lee’s talk on test automation strategy, the client hired Forte Group to develop a strategy for improving their regression testing.
After a brief assessment to gather an understanding of the current activities, we worked with the client to adjust their approach. First, we shared techniques to help the client define regression testing scope and focus based on the context of the system and changes being tested (versus the indiscriminate execution of all existing tests).
Then we introduced the concept of testing versus checking. If testing is viewed as a collection of exploratory and experimental activities, it can be understood as an intrinsically human activity and therefore difficult to fully automate. Checking, on the other hand, represents a process that can be distilled into rules and a possible candidate for automation.
Appreciating this distinction allowed the client to see more clearly why some of their testing activities did not become more efficient with their initial approach to automation. As they revisited their automation strategy, they identified the checks that were appropriate for automation, while applying true testing techniques based on the changes included in each release candidate.
Top Obstacles
Unrealistic Expectations
The client attempted to automate 100% of their existing regression tests without regard to the effectiveness of the tests or the appropriateness to automate them.
Maintenance Challenges
The client found they were spending more time troubleshooting and maintaining their automated test suite than they had previously spent on testing.
Outcome
By taking a new approach to regression testing of their Salesforce-based app, our client reduced regression test effort by almost 50% once the new processes were in place.
The client continued to perform important testing activities manually, but in a more strategic and focused manner. The automation strategy focused on implementing a system of checks using the Forte Group Automation Framework.
The framework has built-in support for Salesforce apps, making it easy to handle Salesforce-specific testing challenges like Lightning UI element identification and the monitoring of API limits during execution.
In addition to removing the regression test bottleneck and enjoying the benefits of a more targeted test approach, our client gained a better understanding of regression testing as a discipline. This allowed the Director of Quality Assurance to better articulate the value it provided, as well as justify the time and effort required.
Key Benefits
More Focused Testing
The client now executes tests in a more focused manner, using appropriate automated checks and adapting their test approach based on the nature of the changes being introduced.
Faster Time to Market
The client saw a significant reduction in regression test cycle time.
Continuous Learning
We implemented a feedback loop that ensured the team continued to learn about the system and continually refine the automated checks as the system evolved.
