Senior Data Engineer - Lakehouse Migrations & AI Pipelines

UnitedHealth Group; Raleigh (NC); Hybrid; USD 92,000 - 164,000; Full time

Job summary

UnitedHealth Group's Optum Insight seeks a Senior Data Engineer to drive a hybrid migration from a legacy data warehouse to a cloud-based lakehouse. You will design ingestion patterns, build scalable data-sharing solutions, and automate pipelines using Azure Data Factory, Databricks, and dbt.

You will read and modify Java and Python code, collaborate with cross-functional teams, and apply AI-assisted development tools to boost productivity while ensuring security, scalability, and performance in

Qualifications

• 5+ years of data engineering or software development experience.
• 3+ years Python or Java programming.
• 2+ years building data pipelines in Azure (ADF, Databricks).
• 1+ year AI-assisted development tools experience.

Responsibilities

• Design diverse data ingestion patterns to migrate legacy Cornerstone Data Warehouse assets into the Magnus Lake House.
• Develop scalable data-sharing solutions for consumer access to data.
• Build automated data pipelines using Azure Data Factory and Databricks.
• Utilize dbt for data transformations following Medallion Architecture (Bronze, Silver, Gold).
• Maintain and enhance Java and Python codebases for project needs.
• Evaluate AI-assisted development trends to inform design decisions.
• Collaborate with cross-functional teams to ensure security, scalability, and performance.

Skills

Data engineering

Python/Java

Azure data services

AI-assisted development tools

SQL/ETL

Education

Bachelor's degree in CS/Math/related field

4+ years data engineering experience (alt with HS diploma)

Tools

Azure Data Factory

Databricks

dbt

GitHub Copilot/Codex

Job description

UnitedHealth Group's Optum Insight seeks a Senior Data Engineer to drive a hybrid migration from a legacy data warehouse to a cloud-based lakehouse. You will design ingestion patterns, build scalable data-sharing solutions, and automate pipelines using Azure Data Factory, Databricks, and dbt.

You will read and modify Java and Python code, collaborate with cross-functional teams, and apply AI-assisted development tools to boost productivity while ensuring security, scalability, and performance in