Senior Data Engineer - NBA

Humana Inc; Nashville (TN); Hybrid; USD 118,000 - 162,000; Full time

Job summary

Humana Inc. is seeking a Senior Data Engineer to build and evolve the data foundation powering the NBA platform on the Databricks Lakehouse.

You will design scalable pipelines, feature workflows, and real-time ingestion to transform healthcare and related data into trusted assets for decision intelligence. This hands-on role focuses on delivering reliable, well-governed data products while partnering with Data Science, AI Engineering, and Platform Engineering teams.

Qualifications

• 5+ years of data engineering experience building production data platforms.
• Strong SQL and Python with hands-on Spark-based data pipelines.
• Experience with Databricks, Delta Lake, or comparable lakehouse platforms.
• Experience implementing Medallion Architecture (Bronze/Silver/Gold) data patterns.
• Experience building batch and streaming data processing pipelines.
• Strong understanding of data modeling, pipeline testing, data quality controls, and operational support practices.
• Familiarity with modern cloud-based data platforms and distributed data processing systems.
• Strong communication skills and the ability to collaborate across engineering, analytics, and business teams.

Responsibilities

• Data pipeline design and maintenance for Bronze, Silver, and Gold lakehouse layers.
• Develop Gold-layer feature tables and reusable data products for model training and real-time decisioning.
• Build batch and real-time ingestion pipelines using Databricks, Spark, and Delta Lake.
• Implement data quality validation, monitoring, and testing controls.
• Design scalable data models balancing governance and performance.
• Tune Spark workloads and storage for efficiency and cost management.
• Ensure data lineage, security, PHI handling, and regulatory compliance.
• Collaborate with DI, AI Engineering, and Platform Engineering teams to enable data availability.
• Diagnose and resolve pipeline issues and production incidents.

Skills

SQL

Python

Spark

Databricks

Delta Lake

Medallion Architecture

Batch & streaming processing

Data modeling

Data governance

Communication

Tools

Databricks

Delta Lake

Job description

Become a part of our caring community

The Senior Data Engineer builds and evolves the data foundation that powers the NBA platform on the Databricks Lakehouse. This role is responsible for developing scalable data pipelines, feature engineering workflows, and real‑time ingestion patterns that transform raw healthcare, member, behavioral, engagement, and socioeconomic data into trusted assets for decision intelligence, machine learning, reinforcement learning, and AI workloads. This is a hands‑on engineering role focused on delivering reliable, well‑governed, and high‑quality data products while partnering closely with Data Science, AI Engineering, and Platform Engineering teams.

Key Responsibilities

• Data pipeline development — Design, build, and maintain Bronze, Silver, and Gold lakehouse pipelines that power member profiles, clinical signals, engagement data, and decision intelligence use cases.
• Feature engineering — Develop and maintain Gold-layer feature tables and reusable data products that support model training, model scoring, reinforcement learning, and real‑time decisioning.
• Batch and streaming ingestion — Build and support batch and real‑time ingestion pipelines using Databricks, Spark, Delta Lake, and event‑driven data architectures.
• Data quality engineering — Implement data quality validation, reconciliation, monitoring, alerting, and testing controls to ensure trusted and production‑ready data assets.
• Data modeling — Design scalable data models that balance usability, governance, performance, and long‑term maintainability.
• Performance optimization — Tune Spark workloads, storage strategies, partitioning schemes, and query performance to improve efficiency, scalability, and cost management.
• Data governance — Follow established standards for data lineage, security, PHI handling, auditability, and regulatory compliance.
• Platform integration — Partner with Decision Intelligence, AI Engineering, and Platform Engineering teams to ensure high‑quality data is available across the NBA ecosystem.
• Operational support — Diagnose and resolve pipeline failures, data quality issues, and production incidents to maintain reliable platform operations.

Use your skills to make an impact

Required Qualifications

• 5+ years of data engineering experience building and operating production data platforms.
• Strong SQL and Python skills with hands‑on experience developing Spark‑based data pipelines.
• Experience with Databricks, Delta Lake, or comparable lakehouse platforms.
• Experience implementing Medallion Architecture (Bronze/Silver/Gold) data patterns.
• Experience building batch and streaming data processing pipelines.
• Strong understanding of data modeling, pipeline testing, data quality controls, and operational support practices.
• Familiarity with modern cloud‑based data platforms and distributed data processing systems.
• Strong communication skills and the ability to collaborate across engineering, analytics, and business teams.

Preferred Qualifications

• Experience with Databricks Feature Store, Unity Catalog, Delta Live Tables, and Databricks Workflows.
• Experience with Spark Structured Streaming and real‑time feature engineering.
• Experience with Kafka and event‑driven data architectures.
• Familiarity with machine learning, recommendation systems, reinforcement learning, or decision intelligence platforms.
• Experience with Azure Data Factory, Azure Data Lake Storage, Azure Event Hubs, or similar cloud‑native data services.
• Experience with data observability and quality platforms such as Great Expectations, Monte Carlo, or equivalent tools.
• Experience integrating external healthcare, claims, CMS, CDC, consumer, or social determinants of health datasets.
• Background in healthcare, insurance, or another regulated industry with PHI/HIPAA handling requirements.

We build with modern AI development tools (such as Claude and GitHub Copilot) and expect everyone on the team to use them to work faster and at higher quality.

Work Style

Remote/Hybrid - Preferably Boston, MA.

Occasional travel to Humana's offices for training or meetings may be required.

Work Hours

Typical business hours are Monday-Friday, 8 hours/day, 5 days/week-- some flexibility might be possible, depending on business needs.

Very minimal travel might be required for training, meetings, and/or conferences

Interview Format

As part of our hiring process, we will be using on-demand technology provided by Hire Vue, a third‑party vendor. This technology provides our team of recruiters and hiring managers with an enhanced method for decision‑making through on‑demand candidate assessments.

If you are selected to move forward from your application prescreen, you will receive correspondence inviting you to participate in an on‑demand assessment with pre‑determined questions. You should anticipate the assessment to take approximately 10‑15 minutes.

Your on‑demand assessment will be reviewed, and you will subsequently be informed if you will be moving forward to next round of interviews.

SSN Task via Workday

Should you be extended a formal employment offer you will receive a request to enter your SSN into our Workday system to scan for duplicate profiles.

Work at Home Requirements

Work at Home Requirements: To ensure Home or Hybrid Home/Office employees’ ability to work effectively, the self-provided internet service of Home or Hybrid Home/Office employees must meet the following criteria: At minimum, a download speed of 25 Mbps and an upload speed of 10 Mbps is required; wireless, wired cable or DSL connection is suggested. In certain roles, the minimum recommended internet speed required by Humana may not be sufficient for business needs. Humana reserves the right to require associates to upgrade their internet service if necessary. Work from a dedicated space lacking ongoing interruptions to protect member PHI / HIPAA information.

Travel: While this is a remote position, occasional travel to Humana's offices for training or meetings may be required.

Scheduled Weekly Hours

40

Pay Range

The compensation range below reflects a good faith estimate of starting base pay for full time (40 hours per week) employment at the time of posting. The pay range may be higher or lower based on geographic location and individual pay will vary based on demonstrated job related skills, knowledge, experience, education, certifications, etc.

$117,600 - $161,700 per year

This job is eligible for a bonus incentive plan. This incentive opportunity is based upon company and/or individual performance.

Description of Benefits

Humana, Inc. and its affiliated subsidiaries (collectively, “Humana”) offers competitive benefits that support whole‑person well‑being. Associate benefits are designed to encourage personal wellness and smart healthcare decisions for you and your family while also knowing your life extends outside of work. Among our benefits, Humana provides medical, dental and vision benefits, 401(k) retirement savings plan, time off (including paid time off, company and personal holidays, paid parental and caregiver leave), short‑term and long‑term disability, life insurance and many other opportunities.

About us

About Humana: Humana Inc. (NYSE: HUM) is a leading U.S. healthcare company. Through our Humana insurance services and our CenterWell healthcare services, we make it easier for the millions of people we serve to achieve their best health – delivering the care and service they need, when they need it. These efforts are leading to a better quality of life for people with Medicare and Medicaid, families, individuals, military service personnel, and communities at large. Learn more about what we offer atHumana.comand atCenterWell.com.

Equal Opportunity Employer

It is the policy of Humana not to discriminate against any employee or applicant for employment because of race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or protected veteran status. It is also the policy of Humana to take affirmative action, in compliance with Section 503 of the Rehabilitation Act and VEVRAA, to employ and to advance in employment individuals with disability or protected veteran status, and to base all employment decisions only on valid job requirements. This policy shall apply to all employment actions, including but not limited to recruitment, hiring, upgrading, promotion, transfer, demotion, layoff, recall, termination, rates of pay or other forms of compensation and selection for training, including apprenticeship, at all levels of employment.

Humana complies with all applicable federal civil rights laws and does not discriminate on the basis of race, color, national origin, age, disability, sex, sexual orientation, gender identity or religion. We also provide free language interpreter services. See our https://www.humana.com/legal/accessibility-resources?source=Humana_Website.