Remote Principal Data Engineer - Azure, Spark and AI

UnitedHealth Group; Eden Prairie (MN); Remote; Confidential; Full time

Job summary

Optum Insight Engineering AI within UnitedHealth Group seeks a Principal Data Engineer to design and build scalable Azure cloud data systems, ETL/ELT pipelines, and AI integrations. Lead data engineering initiatives across Databricks, Snowflake, and Spark to enable ML, analytics and automation.

This role offers remote work within the U.S. (Central hours) with some in-office requirements in Minneapolis or DC four days a week.

Qualifications

• Bachelor's degree or equivalent experience.
• 10+ years of data engineering, data integration, data modeling, data architecture, and ETL/ELT.
• 7+ years of Python experience.
• 5+ years of Apache Spark (PySpark/Spark SQL).
• 5+ years of SQL experience with complex schemas and optimization.
• 3+ years of API design and lifecycle management (GraphQL, REST).
• 3+ years building cloud-based solutions using Azure Databricks with Snowflake/Functions/Service Bus.
• 3+ years Terraform for DevOps automation.
• 3+ years CI/CD tools (GitHub Actions, Git, Artifactory, Sonar).
• 2+ years building LLM integrations for workflow automation.

Responsibilities

• Design and Develop Scalable Cloud Applications in Azure.
• Build and Optimize Large-Scale Data Pipelines on Azure Databricks and Spark.
• Deploy AI-powered data solutions, dashboards, and data products.
• Develop AI-based solutions to automate processes and improve workflows.
• Evolve data engineering architecture and establish best practices.
• Write advanced SQL for data reliability, integrity, and performance.
• Mentor data engineers and provide technical direction.
• Collaborate with stakeholders and external teams on requirements and estimates.
• Evaluate emerging trends to guide design and innovation.
• Ensure scalable, secure, and compliant data platforms.

Skills

Leadership & mentoring

Stakeholder management

Analytical thinking

Education

Bachelor's degree (or equivalent)

Tools

Python

Apache Spark

SQL

API design

Azure Databricks

Snowflake

Terraform

GitHub Actions

Git

Sonar

Azure Functions

Service Bus

Job description

Optum Insight Engineering AI within UnitedHealth Group seeks a Principal Data Engineer to design and build scalable Azure cloud data systems, ETL/ELT pipelines, and AI integrations. Lead data engineering initiatives across Databricks, Snowflake, and Spark to enable ML, analytics and automation.

This role offers remote work within the U.S. (Central hours) with some in-office requirements in Minneapolis or DC four days a week.