The world is full of Patterns (good behaviors) and Anti-Patterns (bad behaviors).
Below we list a number of the “good ones” that your organization should be promoting.
- Success Pattern: Treat data like you would treat code. Don’t exclude data because it’s complex. Automate data tasks like data fabrication or ETL (Extract, Transform and Load) and attach them to your delivery chain.
- Success Pattern: Use Masked “Production-Like” Data. Ensure developers and testers have a rich set of test data to play with. The best source is always production itself. However, ensure the methods you use to Extract, Transform (Mask) and Load support privacy industry regulations.
- Success Pattern: Encrypt or Mask. Use Encryption or Masking methods on vulnerable data e.g. Personally Identifiable Information (PII). Consider this as an opportunity to understand/recognize information risks and consider re-architecting production data for the future.
- Success Pattern: Refresh Data Continuously. Implement a regular automation refresh capability. Data is typically backed up daily (to SANs and/or a Fail-over site). Consider utilizing these copies as a way of obtaining good data without disrupting the production process.
- Success Pattern. PMV Test Data (Profile, Mask and Validate). Introduce Data Compliance across your Non-Production Data. Consider risk profiling methods so you understand risks, remediation methods like masking and validation methods to ensure and prove your compliant end to end.
References:
- DZone DataOps Anti-Patterns Article.