Validating CSV Data
You can use the Task Manager to validate your CSV input data before loading it into PTC Arbortext Content Delivery. It is recommended that you validate your CSV data, before attempting to load the data into the system.
The following things are verified by default as part of the validation process:
• All the files in the input bundle are UTF-8 compliant.
• The input files contain all mandatory columns expected in the structure.
• All of the mandatory columns contain content.
• All of the files in the input bundle do not contain any empty lines in the middle or end of file.
• All of the files in the input bundle do not contain any duplicate entries (entries with the same content across all columns).
• The context or collection is specified in the CSV bundle (either in properties.csv or toc.csv files).
• The context and collection identifiers specified in the properties.csv file exist in the pre-defined list of collections.
• The creation date specified in the properties.csv file is not older than the previously loaded bundle.
• Each of the file name entries specified in the mapping files (such as graphics.csv) exists in the respective folder (such as GRAPHIC) in the bundle.
• Any Image Code entry that is specified in productheirarchy.csv, toc.csv, assembly.csv, or part.csv files exists in Image.csv.
• Any Part Code entry that is specified in assembly.csv exists in part.csv.
• The Applicability column expression contains valid a closing bracket for every opening bracket.
• The Applicability column expression contains valid quote syntax (single quote).
• The Applicability column expression contains valid keyword syntax (used in the right position).
• The Applicability column expression contains valid keywords.
• The Applicability column expression contains valid Date Effectivity syntax.
• The Applicability column expression contains valid PRODUCT values.
• Every Document Code in the toc.csv file has a respective IE reference in the mapping file loaded in the same bundle.
• All PartsList line items in the assembly.csv file have the respective part reference in the part.csv file (in the bundle).
Your system might have other validations defined. Some of the validations produce errors, which prevent the data from being loaded. Some of the validations produce warnings, which do not prevent data loading. In either case, the process produces a CSV file with details about the validation errors or warnings found. No file is produced if the validation succeeds.
Follow these steps to validate your CSV data:
1. Login into Task Manager.
2. Click the Validate data task.
The Validate Data window opens.
3. Enter the desired values for the Upload Location and Input Path parameters.
If the folder containing your bundle is on your local machine, select None for the Upload Location and enter the path to the folder in Input Path.
If the compressed .zip file containing your bundle is on a remote machine, select Local for the Upload Location. In this case, a Browse button is displayed. Click the button to open the Choose File to Upload window where you can browse to the remote file location. Click on the bundle file in the remote location and the path is entered into Input Path. Data Type specifies the input data is SIM data or CSV data.
4. Click Ok to execute the task.
5. Click the Task Manager Monitor Tasks tab to check the progress of the validation and to view the result.
The Validate data task appears in the list of tasks on the Monitor Tasks tab. When the task completes and there are warnings or errors in the results, the View validation results link appears in the comment for the tasks. Click this link to open the validation report.
If you want more details about the results, click the
option next to the task name to open the list of sub-tasks. The validation results appear in the comment for the
CSV Validation Scanner sub-task. The following three results are possible:
• Validation produces no errors or warnings.
In this case, the comment is CSV bundle validation Completed. No output file is produced.
• Validation produces only warnings.
In this case, the comment is CSV data process validation found only warnings. See the file: INS_WORK/Applications/TaskManager/Work/TaskReport/CSVValidationReports/validation_results_taskID.csv.
• Validation produces errors.
In this case, the comment is CSV data process validation failed. See the file: INS_WORK/Applications/TaskManager/Work/TaskReport/CSVValidationReports/validation_results_taskID.csv.
The CSV report file provides a description of the issue, the name of the file containing the issue, the row and column in the file containing the issue, and a suggested resolution. Following is an example of the default output file produced by the validation process:
You can also run a data validation task from the
Transform and Load CSV Data task. In this case, set the
Validate input data parameter to
true. Refer to
Transform and Load (TAL) Tasks for more information about the CSV validation tasks.
