Lead Data Engineer: Azure Databricks & ETL Leader

Octapharma Plasma, Inc.; Charlotte (NC); On-site; USD 120,000 - 190,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401k retirement plan

Paid time off

Tuition Reimbursement

Job summary

Octapharma Plasma, Inc. is seeking a Lead Data Engineer to design and optimize scalable data pipelines using Azure Databricks, PySpark, SQL, and Delta Lake. You will drive data quality, governance, and secure data management while mentoring a team of engineers.

You will collaborate with stakeholders to translate data needs into robust solutions and participate in on-call rotations as required. This is an in-office role based in Charlotte, NC.

Qualifications

• Three years of experience with Azure Databricks platform.
• Five years of data warehouse analysis and design experience.
• Five years of ETL/ELT development, Python, PySpark.
• Python, SQL, and Terraform experience.

Responsibilities

• Lead design, development, and optimization of scalable data pipelines and ETL/ELT workflows.
• Enhance CI/CD pipelines and Azure DevOps, Git, and deployment frameworks.
• Ensure data quality and consistency through integration and cleansing.
• Mentor a team of data engineers and lead best practices.
• Collaborate with stakeholders to translate data needs into technical solutions.
• Assist with on-call rotation and after-hours issues.

Skills

Azure Databricks

PySpark

Python

SQL

ETL/ELT

Data modeling

Data governance

CI/CD

Unity Catalog

Azure RBAC

Education

Bachelor’s degree in computer science or information systems

Tools

Azure Databricks

Terraform

Power BI

MicroStrategy

Azure DevOps

Job description

Octapharma Plasma, Inc. is seeking a Lead Data Engineer to design and optimize scalable data pipelines using Azure Databricks, PySpark, SQL, and Delta Lake. You will drive data quality, governance, and secure data management while mentoring a team of engineers.

You will collaborate with stakeholders to translate data needs into robust solutions and participate in on-call rotations as required. This is an in-office role based in Charlotte, NC.