Data Engineer

MAS Global Consulting, LLC; Colombia; Remote; COP 318,299,000 - 445,619,000; Full time

Job summary

MAS Global Consulting, LLC is seeking a hands-on Data Engineer to build a governance data platform on Microsoft Fabric, covering ingestion, master hub, and back-synchronization to source systems. You will collaborate with a Data Architect, Power BI Developer, and client data stewards.

You will implement a medallion architecture, develop a PySpark-based matching engine, and ensure data quality across Workday, payroll, ERP, and security systems.

Qualifications

• 5+ years in data engineering, including 2+ on the Microsoft data stack.
• Hands-on Microsoft Fabric experience, or deep Synapse or Databricks on Azure plus active Fabric adoption.
• Built a medallion architecture in production.
• Strong PySpark, Spark SQL and T-SQL.
• Entity resolution or MDM: fuzzy matching, survivorship, golden records.
• API integration (REST, OAuth, files, SFTP) and write-back to operational systems.
• Git and CI/CD for data workloads.

Responsibilities

• Build metadata-driven ingestion from source systems into bronze.
• Standardize names, nicknames, suffixes, dates and codes in silver with data quality checks.
• Implement the matching engine in PySpark: shared-ID, exact rules, probabilistic scoring.
• Build the master hub in Fabric SQL: golden records and survivorship.
• Build the stewardship workflow: review queue and audit trail.
• Build the sync back of create, update and deactivate events through Fabric pipelines or MuleSoft.
• Publish gold tables and data quality findings ready for Power BI Direct Lake.
• Schedule and monitor daily runs with secure handling of sensitive data.
• Work with Git and deployment pipelines, and write tests for matching rules.

Skills

Data engineering

Microsoft Fabric

PySpark

Spark SQL

T-SQL

Entity resolution / MDM

API integration

Git

CI/CD

Medallion architecture

Azure Synapse/Databricks on Azure

Tools

Git

CI/CD tooling

Job description

About the project. Our client, a US-based industrial manufacturer, runs on more than a dozen systems with no shared keys between them. MAS Global is building a governed data platform on Microsoft Fabric with two parts: a medallion lakehouse for analytics in Power BI, and a master data hub that keeps one golden record per key entity and syncs corrected values back to the operational systems. The first domain is Workforce: one trusted employee record across Workday HCM, payroll, Epicor ERP and the physical security system.

About the role. You will be the hands-on engineer who builds the platform end to end, from ingestion to the master hub and the integrations that send master updates back to each system. You will work with the Data Architect, a Power BI Developer, and the client's system owners and data stewards.

Responsibilities

• Build metadata-driven ingestion from Workday, payroll, Epicor and security into bronze.
• Standardize names, nicknames, suffixes, dates and codes in silver, with data quality checks.
• Implement the matching engine in PySpark: shared-ID and exact rules, probabilistic scoring such as Jaro-Winkler, and blocking. Tune it against known cases.
• Build the master hub in a Fabric SQL database: golden records, cross-reference IDs, survivorship from rules tables, and lineage.
• Build the stewardship workflow: a review queue, steward edits and a full audit trail.
• Build the sync back of create, update and deactivate events through Fabric pipelines or MuleSoft, with retries and delivery confirmation.
• Publish gold tables and data quality findings, ready for Power BI Direct Lake.
• Schedule and monitor the daily runs, and handle personal and compensation data securely.
• Work with Git and deployment pipelines, and write tests that protect the matching rules.

Qualifications

Required

• 5+ years in data engineering, including 2+ on the Microsoft data stack.
• Hands-on Microsoft Fabric experience, or deep Synapse or Databricks on Azure plus active Fabric adoption.
• Built a medallion architecture in production.
• Strong PySpark, Spark SQL and T-SQL.
• Entity resolution or MDM: fuzzy matching, survivorship, golden records.
• API integration (REST, OAuth, files, SFTP) and write-back to operational systems.
• Git and CI/CD for data workloads.

Nice to have

• Workday, payroll or Epicor data extraction.