This involves dedicating data objects—such as certain accounts, products, or even an entire warehouse—for use by a specific automated test suite. This helps address issues associated with data changes in shared environments.
Here you move data in a known state (usually from production) to a test environment. While this technique is extremely common, it is not necessarily the most effective one—especially in a DevOps environment. Typically, the time required to perform even a partial refresh forces a cadence that is too slow for DevOps, and performing a synchronized reset across integrated systems/components can be complex and error-prone. In instances where a data refresh is feasible, you can make it more effective by combining it with data partitioning. This involves seeding dedicated test data in production so it’s available after each refresh.
Test suite techniques
The automated test suite can create/update the data it needs as a setup task before execution. This technique can be combined with data partitioning to further remediate test data issues. For example, a test might create inventory, but the SKU and inventory location are designated for automated test suite use only. This technique works best in situations where you can use APIs (or even direct data access) to adjust the data. Data setup through the user interface can be a slow process; it also suffers from the same reliability issues as any other UI-level test automation.
Similar to pre-test data creation, this technique uses post-test tasks to reset (i.e., add, modify, delete) data to a known state. As with pre-test data setup, you should perform this technique through API/data access whenever possible.
- On-demand data generation
This involves adding a feature to your automated test framework to recognize data generation tags as opposed to hard-coded data values. This is useful in situations where test data must come from a specific timeframe on the current date. During execution, the framework generates the appropriate data based on the data tag, versus requiring that you maintain test cases before every execution.
This is similar to on-demand data generation but uses static data tags that represent a required data value that changes over time. You can store the current value of each static data tag in a central location where the framework can retrieve it at runtime.
Use test doubles (e.g., stubs, mocks, or virtualized services) in situations where it is difficult (or impossible) to control the data a test needs. This technique works best when there is a clear endpoint that you can replace with a test double, and can be more difficult to use with legacy or monolithic applications.
- Test data management systems
These are full-service test management tools that address several test data-related issues, including data generation, data masking, data sub-setting, and provisioning. Several commercial and open-source tools address parts of the test data management problem. While these tools are extremely powerful, their applicability to specific environments and technologies varies from one tool to another.