Data For Testing
Tests must be a true indicator of market readiness. That requires that the conditions of the test faithfully represent the real world use of the product. That includes the test data. If the test data is not highly similar to real world data, then the tests might pass but the product will be released with unknown defects.
Today it is commonplace for test data to be managed in an ad hoc manner. Agile frameworks say little about data, and so it is not surprising that Agile teams tend to treat data as an afterthought. One of us once heard a development lead chuckle that all their tests pass because they create data that they know will pass.
Test data needs to be curated and managed as a critical development resource. Stakeholders who own business data need to be partners in obtaining and preparing datasets that are suitable for testing. All too often it is the development teams who are left to forage for data that can be used for testing.
Production-like test data also often needs to have sensitive fields “surrogated”. And data used for business intelligence or machine learning might have biases, which cannot be used in many real world situations. See Defining a Data Strategy.
Definition of Ready
A definition of ready (DOR) is a set of criteria that development teams need to be met before they feel they can begin working on something. It is important that a DOR not attempt to define all the things needed to complete work, but rather that it focus on what is needed to ensure that,
All parties are doing their part.
All things that are needed are “underway”—being worked on so that they are provided.
Expecting everything teams need to be in place at the start is unreasonable. Rather, the intention is to ensure that everyone is in motion, working to provide what is needed.
Capability Level
Identify who will provide test data, and the general nature of that data, and “how much is sufficient”.
Feature
Some of the data needed to validate the feature has been provided. Delivery of remaining data should be viewed as part of DOD—completion cannot be expected unless sufficient data has been available for some time.