Setting Up Data Sync
Set up full and incremental data sync pipelines in Matillion to load data from external applications into PTC Orbit through Snowflake.
Matillion performs full and incremental data syncs from external applications to PTC Orbit stage objects stored in Snowflake. This topic covers enabling data ingestion modes on objects, configuring orchestration and transformation pipelines for Salesforce data, syncing from AWS S3, creating custom connectors, and importing pre-built pipelines.
Ingestion Modes
PTC Orbit stage objects support three ingestion modes:
• Staging Review: Data is staged for user review before committing to the database. Incoming records are evaluated against Match rules and grouped into Unified Records for a reviewer to approve. For example, Asset object.
• Direct Sync: Data loaded by Matillion pipelines is pulled directly from Snowflake into the production database on a scheduled or on-demand basis, without staging or review. For example, Work Order object.
Perform the following steps to configure data sync.
1. Log in to the customer PTC Orbit tenant.
2. In Max Designer, click Objects on the Developer Tools launchpad menu. In the list view, click the object record to configure.
3. Set the Data Ingestion Mode field to Staging Review or Direct Sync, then click Save.
4. In the Pending Metadata Changes dialog, review the list of affected fields and objects, then click Confirm and Apply.
5. Wait for the database consistency check notifications to confirm the metadata update is complete: Database consistency check started followed by Database consistency check completed.
6. On the Development Actions launchpad menu, click Synchronize to apply the change.
7. If you set Data Ingestion Mode to Staging Review, open the Rules page to confirm the object is available for Match rules.
The object is ready for data ingestion through Matillion pipelines.
What to Do Next
To configure the preventive maintenance data pipeline that extracts PM data from Salesforce through Matillion into Snowflake, see
Configuring the Maintenance Data Pipeline.
System Jobs for Data Sync
PTC Orbit uses two system jobs to synchronize staging records from Snowflake to the PTC Orbit PostgreSQL database:
• Pull Pre-DB Sync Records from Snowflake: Syncs Pre-DB Sync records with a status of New or Failed to Sync from Snowflake to their corresponding business object tables.
• Sync Unified Records: Syncs Data Review Stage Unified records with a status of Auto Mapped, Reviewed, or Sync Failed from Snowflake to the PTC Orbit database.
To run the
Pull Direct Sync Records from Snowflake job manually, see
Running a Direct Sync System Job.