Remote Principal Data Engineer: Cloud, AI & Pipelines

UnitedHealth Group; Eden Prairie (MN); Remote; USD 113,000 - 193,000; Full time

Benefits offered by this job

Comprehensive benefits package

Equity stock purchase

401k contribution

Telecommuter policy

Job summary

UnitedHealth Group's Optum Insight Engineering AI team seeks a Principal Data Engineer to design and develop scalable Azure-based data systems, ETL/ELT pipelines, and Spark workflows that power machine learning, analytics, and business automation. You will mentor engineers, collaborate with stakeholders, and shape data architecture while ensuring security, privacy, and reliability across the platform.

Remote work is supported within the U.S.; some roles require onsite four days weekly in certain

Qualifications

• Bachelor's degree or equivalent experience in data engineering or related field.
• 10+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes.
• 7+ years of experience in Python.
• 5+ years of experience in Apache Spark (PySpark/Spark SQL).
• 5+ years of experience in SQL, including designing complex data schemas and query performance optimization.
• 3+ years of experience with API design and lifecycle management (GraphQL, REST, etc.).
• 3+ years of experience building and deploying cloud-based solutions using Azure Databricks with UC, Snowflake, Functions, or Service Bus.
• 3+ years of experience with DevOps automation using Terraform.
• 3+ years of experience with CI/CD processes and tools (GitHub Actions, GIT, Artifactory, or Sonar).
• 2+ years of experience building LLM integrations for workflow automation or business needs.

Responsibilities

• Design and Develop Scalable Cloud Applications: Develop services, controls, and reusable patterns in the Azure public cloud.
• Build and Optimize Large-Scale Data Pipelines: Design, build, optimize, and manage modern ETL/ELT processing on Azure Databricks and Spark.
• Deploy AI and Data-Driven Solutions: Develop and deploy data pipelines for ML, BI dashboards, and AI tools.
• Develop AI-Powered Business Solutions: Build AI-based solutions to automate processes and improve workflows.
• Architectural Evolution and Standards: Define data engineering standards and best practices for data collections and integrations.
• Improve System Quality and Data Reliability: Write advanced SQL with performance tuning to improve data reliability and efficiency.
• Collaborative Leadership and Mentoring: Mentor engineers and provide technical direction on cloud data platforms.
• Stakeholder and Requirement Analysis: Understand product capabilities, decompose implementations, and provide realistic estimates.
• Evaluate Emerging Trends: Inform solution design and strategic innovation for cloud data architectures.
• Best Practices in performance scalability and optimization.

Skills

Python

Apache Spark

SQL

Azure Databricks

Data pipelines

LLM integrations

API design

Terraform

CI/CD

Cloud data platforms

Education

Bachelor's degree or equivalent experience

Tools

GitHub Actions

Artifactory

Sonar

Job description

UnitedHealth Group's Optum Insight Engineering AI team seeks a Principal Data Engineer to design and develop scalable Azure-based data systems, ETL/ELT pipelines, and Spark workflows that power machine learning, analytics, and business automation. You will mentor engineers, collaborate with stakeholders, and shape data architecture while ensuring security, privacy, and reliability across the platform.

Remote work is supported within the U.S.; some roles require onsite four days weekly in certain