Get started
Insights

Cost Optimization Through Effective Performance Testing Strategies

Cost Optimization Through Effective Performance Testing Strategies

We recently hosted a webcast titled “Cost Avoidance & Optimization through Effective Performance Testing Strategies.” We also hosted “Performance Tour Live Event 2023,” an in-person event in Chicago. At both events I presented material with my good friend Scott Moore, Host of The Performance Tour & The SMC Journal. In this blog post, I provide a summary of what Scott and I covered.

 

We discussed:

 

  • The true value of performance testing
  • Understanding cost implications
  • Building an effective performance testing foundation
  • Shifting performance left and moving towards continuous performance testing
  • Real-world case studies that inspire action

The value of performance testing

I like to compare it to an insurance policy. You can release mission-critical systems without performance testing, but you just don’t know what might happen in production. You could experience system failures and poor performance, which can lead to dissatisfied customers and lost revenue.

Without an insurance policy, you’re flying blind and there’s nothing in place to save you from the unknown. Performance testing is that insurance policy. It can:
  • Avoid costly outages
  • Optimize resource costs
  • Preserve brand value
  • Increase user retention
  • Increase customer conversion
  • Improve employee productivity
  • Improve search engine ranking

Scott noted that performance testing case studies are using data and measurement to help quantify its benefits. For example, on e-commerce sites, every second of reduced processing time can result in 7-10% more conversions (i.e., online sales).

How value is derived from performance testing

Next, Scott and I discussed how value is derived from performance testing. Starting from the top of the pyramid (diagram below), we progress through these steps:

  • Findings
  • Analysis
  • Results
  • Planning, Development and Execution

CostOptimizationPerfTesting_Graphic1

For the “Results” stage, Scott recommends that we state the results in tangible terms, such as cost savings or revenue gained. When you do that, you gain the attention of the C-suite, says Scott. I totally agree. Sometimes we can think about results in terms that are too technical and not focus on business results. We should start with the latter first.

An effective testing foundation

We can ensure value with an effective testing foundation:

CostOptimizationPerfTesting_Graphic2

The steps in the foundation are:

  • Plan, Develop & Execute
  • Results
  • Analyze


Scott says that the planning phase is the most important phase. He encourages us to narrow the scope in this phase and figure out your “what and why.” Ask a series of questions to distill down the list of requirements and objectives. 

Among these three phrases, Scott’s advice to performance engineers is to focus on Analysis. How you communicate the analysis and tell the right story to stakeholders helps you become more valuable to your company. Scott recommends that managers get their teams trained on analysis, as that will create a more effective team.

Continuous performance testing

Next we discussed Continuous Performance Testing.

CostOptimizationPerfTesting_Graphic3I think of Continuous Performance Testing as the evaluation of performance at each stage of the delivery pipeline. It’s frequent and visible performance feedback that allows us to trend performance data across builds.

Scott and I presented these steps:

  • Commit (automated unit performance testing)
  • Integration (automated component performance testing)
  • User Acceptance / Performance Load (system-level performance testing)
  • Production (automated performance testing)

Scope of performance engineering

Finally, Scott discussed the wide scope of performance engineering. He says that it’s so much more than code. Performance engineering applies throughout the entire software lifecycle and starts before a line of code is written. It ends not with testing or production, but with the business actually using the software and deriving value from it. 

Performance engineering sits in the middle and interacts with:

  • Business & Contracts
  • Dev
  • Quality Assurance (Testing)
  • Platform Engineering
  • Monitoring / Observability
  • Capacity Planning
  • Simulation



You may also like...

Adopting Detection-as-Code: Transforming Cloud Security Operations with a Proactive Approach

4 min By Carlos Cortes

Exploring AI in Software Testing

5 min By Lee Barnes

Developers and GenAI: Skills for the Future

3 min By Humberto Bringas
More Insights