Senior Data Engineer - Lakehouse & Real-Time Pipelines

Humana; United States; Híbrido; USD 118 000 - 162 000; Tempo integral

Resumo da oferta

Humana is seeking a Senior Data Engineer to build and evolve the NBA platform data foundation on the Databricks Lakehouse. The role focuses on scalable pipelines, feature engineering, and real-time ingestion for decision intelligence and AI workloads.

You’ll partner with Data Science, AI Engineering, and Platform Engineering teams to deliver trusted, high‑quality data products while ensuring governance and PHI handling compliance.

Qualificações

• Bachelor’s degree in computer science or related field
• 5+ years of data engineering experience building and operating production data platforms
• Strong SQL and Python skills with Spark-based data pipelines
• Experience with Databricks, Delta Lake, or comparable lakehouse platforms
• Experience implementing Medallion Architecture (Bronze/Silver/Gold) data patterns
• Experience building batch and streaming data processing pipelines
• Strong understanding of data modeling, pipeline testing, data quality controls, and operational support practices
• Familiarity with modern cloud-based data platforms and distributed data processing systems
• Strong communication skills and the ability to collaborate across engineering, analytics, and business teams

Responsabilidades

• Design, build, and maintain Bronze, Silver, and Gold lakehouse pipelines for member profiles and signals
• Develop and maintain Gold-layer feature tables for model training and real-time scoring
• Build batch and real-time ingestion pipelines using Databricks, Spark, and Delta Lake
• Implement data quality validation, monitoring, and alerting for production data assets
• Design scalable data models balancing usability, governance and performance
• Tune Spark workloads and storage to improve efficiency and cost
• Ensure data governance, lineage, PHI handling, and regulatory compliance
• Collaborate with Data Science, AI Engineering and Platform Engineering teams

Conhecimentos

SQL

Python

Spark

Databricks

Delta Lake

Modeling

Communication

Formação académica

Bachelor's degree in Computer Science or related field

Ferramentas

Databricks

Delta Lake

Spark

Descrição da oferta de emprego

Humana is seeking a Senior Data Engineer to build and evolve the NBA platform data foundation on the Databricks Lakehouse. The role focuses on scalable pipelines, feature engineering, and real-time ingestion for decision intelligence and AI workloads.

You’ll partner with Data Science, AI Engineering, and Platform Engineering teams to deliver trusted, high‑quality data products while ensuring governance and PHI handling compliance.