Healthcare Databricks Data Engineer: Scalable Pipelines

Gentiva; Atlanta (GA); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Health Insurance

401k Plan

Tuition Reimbursement

PTO

Mileage Reimbursement

Professional growth opportunities

Job summary

Responsibilities include building ETL/ELT pipelines, implementing Delta Lake, governance via Unity Catalog, and CI/CD for data workflows, with collaboration across BI, data analytics, and data science teams.

Qualifications

• Bachelor's degree in a related field.
• 5+ years of data engineering, analytics, or software development experience.
• 3+ years of hands-on experience with the Databricks platform.
• Healthcare IT or healthcare data experience preferred.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory, Databricks and Apache Spark.
• Implement Delta Lake architecture for reliable data storage and processing.
• Build and optimize data workflows using Databricks Workflows and Jobs.
• Develop scalable data models following medallion architecture (bronze, silver, gold).
• Implement Unity Catalog for data governance, access control, and metadata management.

Skills

Python (PySpark)

SQL

Git version control

Team collaboration

Education

Bachelor's degree in CS/IT or related field

5+ years data engineering/analytics experience

3+ years hands-on with Databricks

Tools

Databricks Platform

Azure Data Factory

Apache Spark

Delta Lake

Unity Catalog

Azure DevOps

Power BI/SSRS

Job description

Responsibilities include building ETL/ELT pipelines, implementing Delta Lake, governance via Unity Catalog, and CI/CD for data workflows, with collaboration across BI, data analytics, and data science teams.