Enterprise ERP Data Migration

Engineered a custom Python integration suite to execute a high fidelity migration

Role: Lead Engineer

Timeline: 2024

Enterprise ERP Data Migration
  • Python
  • Shopify
  • Automation
  • Data Architecture
  • Systems Design
  • NodeJS
  • React

The Problem (Why)

Zep Inc. (a major manufacturer of commercial and household cleaning supplies) needed a high fidelity solution for migrating data from their legacy systems.

Their legacy ERP contained:

  • Over 3k individual product SKUs, with 75 data fields per SKU and 30 non-standard fields.
  • More than 10k individual documents (data sheets) fragmented across multiple systems.
  • No way to natively map data to domain-specific data structures, like collections and variants.

The Technical Challenge and Solution (How)

I lead the development of a suite of python tools that would automate data integration, facilitate domain-specific port, and orchestrate an error-free migration. Absolute data integrity was essential, as even a marginal error rate could result in significant financial loss for the business. Our solution required 100% precision.

Validation of Legacy System Data / Export Data Grooming Data exported from their legacy systems contained artifacts, corrupted data, and inaccuracies introduced by human error. Our suite needed to statically analyze the exported data, flag for inaccuracies, and apply patches where applicable. We were able to provide the business high confidence in data “cleanliness”, by enforcing strict rules to our base master data export.

Port Data to Domain-Specific Fields Implemented a mapping layer to port master data to domain-accurate data structures. This required bridging deep knowledge of the new domain and understanding the legacy data’s shape. Since some data did not lacked native representation, we utilized cutting-edge platform features to create custom fields.

Migrate with Auditable Trace Export migration trace artifacts to validate results and verify data integrity. This included dry-run capabilities, an optional manual import document, visual regression validation GUI, and exportable execution logs. With an auditable “paper trail”, we provided the business with the confidence to scale migration efforts and assure zero data loss.

The Result

Zep launched a Shopify 2.0 rebuild and completed a large-scale, high-integrity catalog migration without data loss.

  • Migrated 2,500+ SKUs from legacy ERP into Shopify with strict validation and auditability.
  • Automated SDS/TDS document matching and attachment across product listings, reducing a large manual process to a repeatable CLI workflow.
  • Produced migration trace artifacts (logs, dry runs, optional manual import docs) to enable verification and future scaling of migration efforts.