Data Foundry
Manage the end-to-end data validation pipeline, from staging review and ETL configuration to matching rules and production sync.
Overview
Data Foundry ensures accurate data transfer and validation before records enter the PTC Orbit system. The end-to-end process includes the following stages:
• Enable Staging Review: Enable Staging Review Required in Max to create intermediate staging tables in Snowflake for data preprocessing.
• Create Data Connector: Create a Data Connector in Max to identify the data source for records entering PTC Orbit.
• Configure ETL Pipeline: Configure the ETL pipeline in Matillion to establish a secure connection between external data sources and Snowflake.
• Define Matching Rules: Define matching rules in PTC Orbit to manage duplicate records and determine the need for human intervention.
• Review Data: Review data in PTC Orbit to ensure only clean, accurate data is synced to production.
It has two parts in the PTC Orbit app. First is to create matching rules and the next is to review them that fails validation.
• Rules:
◦ Users can define matching rules to identify duplicates, enforce data consistency, and validate object details.
◦ Rules can be created for different objects such as Assets and Products.
◦ This ensures that only standardized and validated data flows into business processes.
• Data Review:
◦ Users can review flagged records, analyze the cause of failure, and take corrective measures.
◦ This step prevents inaccurate or incomplete data from propagating in the system.
Use cases
• Duplicate Detection: Identify and merge duplicate asset or product records based on configurable rules.
• Error Correction: Review failed records and correct missing or inaccurate details before they impact reporting.
• Process Reliability: Improve the accuracy of downstream workflows such as work orders, campaigns, or scoring by filtering out invalid data.
Related Topics