Lead Data Engineer: Azure Databricks & Pipelines Leader

Associate Staffing; Charlotte (NC); On-site; USD 130,000 - 140,000; Full time

Job summary

Associate Staffing is recruiting a Lead Data Engineer to design and optimize scalable data pipelines in a fast-growing plasma donation business. You will lead ETL/ELT workflows using Azure Databricks, PySpark, SQL and Delta Lake, mentor engineers, and drive data-driven decisions.

Based onsite in Charlotte, NC, you will collaborate with stakeholders, strengthen data governance with Unity Catalog and Azure RBAC, and advance CI/CD pipelines with Azure DevOps.

Qualifications

• Lead design, development, and optimization of scalable data pipelines and ETL/ELT workflows using Azure Databricks, PySpark, SQL, and Delta Lake.
• Drive CI/CD pipelines and automated deployment using Azure DevOps, Git, and automation frameworks.
• Ensure data quality, security, and governance with Unity Catalog, Azure RBAC, and managed identities.
• Lead and mentor a team of data engineers, promoting best practices and technical growth.
• Collaborate with stakeholders to translate data needs into technical solutions.

Responsibilities

• Lead the architecture and implementation of scalable data pipelines.
• Mentor and grow the data engineering team through code reviews and coaching.
• Collaborate with data scientists and analysts to enable data-driven decisions.
• Ensure data integrity, security, and governance across platforms.
• Support on-call duties and assist with incident resolution as needed.

Skills

Azure Databricks

PySpark

SQL

Data pipeline design

Leadership

Data governance

Education

Bachelor's degree in Computer Science or related field

Tools

Azure DevOps

Git

Automation frameworks

Unity Catalog

Azure RBAC

Job description

Associate Staffing is recruiting a Lead Data Engineer to design and optimize scalable data pipelines in a fast-growing plasma donation business. You will lead ETL/ELT workflows using Azure Databricks, PySpark, SQL and Delta Lake, mentor engineers, and drive data-driven decisions.

Based onsite in Charlotte, NC, you will collaborate with stakeholders, strengthen data governance with Unity Catalog and Azure RBAC, and advance CI/CD pipelines with Azure DevOps.