Lead Data Engineer

Octapharma Plasma, Inc.; Charlotte (NC); On-site; USD 120,000 - 190,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401k retirement plan

Paid time off

Tuition Reimbursement

Job summary

Octapharma Plasma, Inc. is seeking a Lead Data Engineer to design and optimize scalable data pipelines using Azure Databricks, PySpark, SQL, and Delta Lake. You will drive data quality, governance, and secure data management while mentoring a team of engineers.

You will collaborate with stakeholders to translate data needs into robust solutions and participate in on-call rotations as required. This is an in-office role based in Charlotte, NC.

Qualifications

• Three years of experience with Azure Databricks platform.
• Five years of data warehouse analysis and design experience.
• Five years of ETL/ELT development, Python, PySpark.
• Python, SQL, and Terraform experience.

Responsibilities

• Lead design, development, and optimization of scalable data pipelines and ETL/ELT workflows.
• Enhance CI/CD pipelines and Azure DevOps, Git, and deployment frameworks.
• Ensure data quality and consistency through integration and cleansing.
• Mentor a team of data engineers and lead best practices.
• Collaborate with stakeholders to translate data needs into technical solutions.
• Assist with on-call rotation and after-hours issues.

Skills

Azure Databricks

PySpark

Python

SQL

ETL/ELT

Data modeling

Data governance

CI/CD

Unity Catalog

Azure RBAC

Education

Bachelor’s degree in computer science or information systems

Tools

Azure Databricks

Terraform

Power BI

MicroStrategy

Azure DevOps

Job description

Want to Expand

your career-development potential,

your ability to help donors and patients,

and your access to professional opportunities?

We’re growing fast.

[You can, too!]

There are so many ways Octapharma Plasma can enhance your life and your career. Our strong growth is creating great learning and career development opportunities throughout our company, and especially our donation centers. Because you’re someone who loves to learn, enjoys people, and has a real heart to help, we encourage you to join us as a:

Lead Data Engineer

• Lead design, development, and optimization of scalable data pipelines and ETL/ELT workflows using Azure Databricks, PySpark, SQL, and Delta Lake
• Enhancement of CI/CD pipelines and Azure DevOps, Git, and automated deployment frameworks to streamline development and release processes
• Ensure data quality and consistency through data integration, transformation, and cleansing
• Familiarity with security and governance best practices using Unity Catalog, Azure RBAC, managed identities, and data access controls to ensure compliance and secure data management is preferred
• Responsibility includes leading a team of data engineers and mentoring in best practices.
• Utilize Python for data manipulation, automation, and custom data solutions
• Collaborate with stakeholders to understand data needs and translate them into technical solutions
• Participate in on-call rotation and assist with after-hours issues, as required
• Perform other duties as assigned

This Is What It Takes

• Bachelor’s degree in computer science or information systems science, or equivalent experience
• Three (3) years experience with Azure Databricks platform
• Five (5) years of comprehensive data warehouse analysis and design experience, with full knowledge of data warehouse methodologies, data modeling and familiarity with operational data stores.
• Five (5) years of experience with ETL/ELT Development, Python, PySpark
• Python, SQL, and Terraform experience

Required Certifications, Licenses, And Registrations

• Databricks Data Engineer Professional or Databricks Data Engineer Associate Certification preferred

Knowledge, Skills, And Abilities

• Ability to adapt to changing priorities and respond to problems creatively
• Ability to assist clients with migrating from legacy to modern data platform
• Proficiency with technical, conceptual, interpersonal, written, and verbal communication skills
• Knowledge of general business, quality and regulatory requirements preferred
• Familiarity with Power BI and MicroStrategy architecture is a plus

Physical Requirements

• Position works in-office at the corporate location
• Ability to sit for extended periods of time
• Ability to use a computer and other office equipment
• Ability to occasionally lift and carry up to 20 pounds
• Ability to reach, bend, and stoop as necessary
• Ability to communicate effectively, both verbally and in writing
• Ability to focus and concentrate on tasks for extended periods
• Ability to navigate the office environment safely, including stairs and elevators (if applicable)
• Ability to travel up to 20% via airplane or vehicle (if applicable)
• Ability to use assistive devices if needed for mobility or communication

Do Satisfying Work. Earn Real Rewards And Benefits

We’re widely known and respected for our benefits and for leadership that is supportive and hands-on. Managers who truly want you to grow and excel.

• Formal training
• Outstanding plans for medical, dental, and vision insurance
• Health savings account (HSA)
• Flexible spending account (FSA)
• Tuition Reimbursement
• Employee assistance program (EAP)
• Wellness program
• 401k retirement plan
• Paid time off
• Company paid holidays
• Personal time