Azure Databricks Migration Lead: ELT & Data Modernization

Machine Learning Technologies, LLC; Wilmington (DE); On-site; USD 120,000 - 180,000; Full time

Job summary

Machine Learning Technologies, LLC is seeking a senior engineer to lead the migration of data pipelines from Informatica to Azure Databricks. The role requires architecting the target state, planning phased execution, and implementing PySpark-based pipelines while modernizing ETL patterns to ELT best practices.

You will provide technical leadership across solution design, code quality, testing strategy, deployment, and production readiness, collaborating with stakeholders to ensure alignment

Qualifications

• Experience migrating data pipelines from Informatica to Databricks.
• Hands-on with PySpark to build scalable pipelines.
• Strong understanding of ELT patterns and modern data engineering.
• Experience with Azure Databricks best practices and Unity Catalog.

Responsibilities

• Lead end-to-end migration of data pipelines, workflows, and integration patterns to Azure Databricks.
• Define migration strategy, target architecture, and phased execution plans.
• Provide technical leadership on design, standards, testing, deployment, and production readiness.
• Mentor engineers on Databricks design patterns and reusable data engineering frameworks.
• Partner with business, architecture, security, and operations to align migration with enterprise priorities.

Skills

Azure Databricks

PySpark

ETL/ELT

Data Pipelines

Cloud Architecture

Tools

Informatica PowerCenter

Oracle

Spark SQL

Job description

Machine Learning Technologies, LLC is seeking a senior engineer to lead the migration of data pipelines from Informatica to Azure Databricks. The role requires architecting the target state, planning phased execution, and implementing PySpark-based pipelines while modernizing ETL patterns to ELT best practices.

You will provide technical leadership across solution design, code quality, testing strategy, deployment, and production readiness, collaborating with stakeholders to ensure alignment