Azure Data Pipeline Engineer – ADF & Databricks Expert

JCW Group; Charlotte (NC); On-site; USD 150,000 - 180,000; Full time

Job summary

A leading data service firm in Charlotte, NC is looking for a Mid-Senior Data Engineer to support Azure Data Factory and Databricks during a significant cloud migration initiative. You will manage ETL/ELT workflows, ensuring their stability and performance while collaborating with engineering teams. The ideal candidate has strong experience with data engineering tools and services, emphasizes troubleshooting and optimization of data pipelines, and thrives in production environments with high responsibilities.

Qualifications

• Strong experience with Azure Data Factory, Databricks, and Azure‑based data engineering.
• Proven background in ETL/ELT processes, data integration, and orchestration.
• Solid understanding of cloud data services (ADLS Gen2, Azure SQL, Databricks, Functions, etc.)
• Hands‑on experience in monitoring, debugging, and optimizing production data pipelines.

Responsibilities

• Support and troubleshoot Azure Data Factory pipelines and Databricks jobs across production environments.
• Ensure stability, performance, and uptime of ETL/ELT workflows during cloud migration initiatives.
• Perform root‑cause analysis on pipeline failures, Spark issues, cluster timeouts, and data mismatches.
• Collaborate with engineering and DevOps teams to optimize workflows, deployments, and cloud integrations.

Skills

Databricks

ETL/ELT processes

Data integration

Tools

Azure Data Factory

Azure SQL

ADLS Gen2

Azure Functions

Job description

A leading data service firm in Charlotte, NC is looking for a Mid-Senior Data Engineer to support Azure Data Factory and Databricks during a significant cloud migration initiative. You will manage ETL/ELT workflows, ensuring their stability and performance while collaborating with engineering teams. The ideal candidate has strong experience with data engineering tools and services, emphasizes troubleshooting and optimization of data pipelines, and thrives in production environments with high responsibilities.