Creating Data Extraction and Format Requirements
Data extraction and format requirements must be clearly communicated to the person doing the extraction. Once the data is extracted, if the validation shows errors or anomalies in the data, return to the process of developing these requirements to eliminate the anomalies.
The following list describes common solutions for issues for data extraction and data formatting. They should be resolved in a requirements document for data extraction.
• Specify terminology in a table that maps the terms among the systems.
• Specify that data should be provided in text-based files readable by common programs such as Notepad, TextPad, WordPad, and so on.
• Specify how files should be named (for example, customer_xxxx_yyyy.txt, where xxxx is the file type (part, BOM, document, and so on).
• Specify how data should be segregated (for example, by object type).
• If the data files will contain additional attributes that need to be calculated or derived, these attributes should be identified.
• Specify how you want attributes (or fields) to be separated (for example, by a tilde (~).
Developing Load Metrics
Once the sample data set has been successfully loaded, you can calculate how long it took to load the data file. This allows you to calculate the total parts-per-minute for loading. From these metrics, you can extrapolate the amount of time required to load all of the data, which may affect scheduling of the load activities.