Promoting Pipelines from Sandbox to Production
Export tested data pipelines from a Matillion sandbox project and import them into the production project.
Matillion projects are set up as sandbox and production pairs for each customer. Develop and test data pipelines in the sandbox environment first. After the pipelines pass all tests and run without errors, export them from the sandbox project and import them into the production project.
PTC Orbit uses two system jobs to synchronize staging records from Snowflake to the PTC Orbit PostgreSQL database:
• Pull Pre-DB Sync Records from Snowflake: Syncs Pre-DB Sync records with a status of New or Failed to Sync from Snowflake to their corresponding business object tables.
• Sync Unified Records: Syncs Data Review Stage Unified records with a status of Auto Mapped, Reviewed, or Sync Failed from Snowflake to the PTC Orbit database.
Perform the following steps to promote pipelines from sandbox to production.
1. Log in to Matillion and open the sandbox project.
2. In the Files panel, hover over the pipeline, folder, or project name at the top of the folder tree. Click the three-dot menu (...) that appears next to it.
3. Click Export. The file downloads as a ZIP archive to your local machine with the naming format: <project name>_<branch>_<file/folder name>_<yyyymmdd>T<hhmmss>.zip.
4. Open the production project in Matillion.
5. In the Files panel, hover over the destination folder. Click the three-dot menu (...) that appears next to it.
6. Click Import. Browse to the exported ZIP file, select it, and click Open. The pipelines are imported into the destination folder.
7. Update the following settings in the imported pipelines to match the production environment:
◦ Login credentials: External application usernames, passwords, and API keys.
◦ Connection settings: Endpoint URLs, Snowflake warehouse defaults, and database references.
◦ Project variables: Create any project variables that the pipelines reference, as these are not included in the exported package.
◦ Schedules: Re-create any pipeline schedules, as these are not included in the exported package.
The pipelines are deployed in the production Matillion project with environment-specific settings applied.
 |
If the destination folder already contains a file or folder with the same name as an item in the import package, delete it before importing to prevent failure.
|
When importing a pipeline folder or an entire project, the imported content becomes a subfolder within the destination. Verify the folder structure after import to ensure pipelines are in the expected location.