Sr Data Engineer

Taleo; Eden Prairie (MN); Remote; USD 92,000 - 164,000; Full time

Benefits offered by this job

Telecommute option

Healthcare benefits

Job summary

Optum is seeking a Senior Data Engineer on the Optum Technology team to design scalable, cloud-native data pipelines using PySpark, Delta Lake, and Azure Data Factory. You will collaborate with Data Science teams to operationalize ML models and ensure data quality, security, and governance across healthcare data.

You can telecommute from anywhere in the U.S. This role requires 4+ years in data engineering with Databricks and Azure, plus CI/CD experience.

Qualifications

• 4+ years of Python and PySpark experience in data engineering.
• 4+ years building scalable ETL/ELT data pipelines with Databricks.
• 4+ years in Agile/Scrum project environments.
• 4+ years hands-on with Azure Databricks and Delta Lake.
• 4+ years CI/CD pipelines using GitHub Actions or Azure DevOps.
• 3+ years Azure cloud services including ADF or ADLS.
• 2+ years with unit testing, data validation, and pipeline testing.

Responsibilities

• Design and develop scalable ETL/ELT data pipelines using Databricks, Apache Spark, and Python on Azure cloud platforms.
• Build cloud-native data solutions leveraging Azure Data Lake Storage, Azure Data Factory, and Delta Lake architecture.
• Collaborate with Data Science and AI teams to operationalize ML models, feature stores, and real-time/streaming frameworks.
• Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI.
• Build reusable frameworks, engineering accelerators, and automation scripts to improve engineering efficiency and delivery quality.
• Ensure data quality, governance, security, and compliance across the complete data lifecycle.
• Optimize the performance, scalability, and cost-efficiency of batch and streaming data workflows, including Databricks Delta Live Tables.
• Participate in Agile development practices and provide production support/on-call coverage to ensure system stability.
• Leverage enterprise-approved AI tools to streamline software engineering workflows and drive continuous improvement.

Skills

Python

PySpark

Databricks

Azure

CI/CD

GitHub Actions

Azure DevOps

Education

Bachelor's degree in Computer Science or related field

Tools

Azure Data Factory

Azure Data Lake Storage

Delta Lake

Databricks

GitHub Actions

Azure DevOps

Job description

Improve the lives of others while Caring. Connecting. Growing together.

Our Company

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, and data they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits, and career development opportunities. Come make an impact on the communities we serve as you help us advance health equity on a global scale. Join us to start Caring. Connecting. Growing together.

Position Summary

As a Senior Data Engineer on the (Input Required) team at Optum Technology, you will lead the design and development of scalable, cloud-native data pipelines and machine learning data architectures on Azure and Databricks. In this role, you will build data integration routines using PySpark, Delta Lake, and Azure Data Factory, while collaborating with Data Science and AI teams to operationalize ML models, feature stores, and streaming analytics frameworks. You will leverage enterprise-approved AI tools to automate workflows, establish reusable engineering accelerators, and ensure data quality, security, and responsible AI implementation across high-volume healthcare data ecosystems.

You’ll enjoy the flexibility to telecommute* from anywhere within the U.S. as you take on some tough challenges.

Primary Responsibilities

• Design and develop scalable ETL/ELT data pipelines using Databricks, Apache Spark, and Python on Azure cloud platforms
• Build cloud-native data solutions leveraging Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), and Delta Lake architecture
• Collaborate with Data Science and AI teams to operationalize machine learning models, feature stores, model input pipelines, and real-time/streaming frameworks
• Design, develop, and deploy AI-powered solutions to address complex business challenges with an emphasis on responsible use of AI
• Build reusable frameworks, engineering accelerators, and automation scripts to improve engineering efficiency and delivery quality
• Ensure data quality, governance, security, and compliance across the complete data lifecycle
• Optimize the performance, scalability, and cost-efficiency of batch and streaming data workflows, including Databricks Delta Live Tables (DLT)
• Participate in Agile development practices (sprint planning, code reviews, CI/CD automation with GitHub Actions or Azure DevOps), and provide production support/on-call coverage to ensure system stability
• Leverage enterprise-approved AI tools to streamline software engineering workflows, automate tasks, and drive continuous operational improvement
• Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI

You’ll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.

Required Qualifications

• 4+ years of experience in Data Engineering developing with Python and PySpark
• 4+ years of experience building scalable ETL/ELT data pipelines using Databricks
• 4+ years of experience working within Agile/Scrum project environments
• 4+ years of hands-on experience with Azure Databricks and Delta Lake architecture
• 4+ years of experience implementing CI/CD pipelines using GitHub Actions or Azure DevOps
• 3+ years of hands-on experience with Azure cloud services, including Azure Data Factory (ADF) or Azure Data Lake Storage (ADLS)
• 2+ years of experience with unit testing, data validation, and pipeline testing frameworks

Preferred Qualifications

• Bachelor's or higher degree in Computer Science, Engineering, Information Technology, or a related field
• Experience building, operationalizing, or deploying machine learning models, feature stores, and AI-powered data pipelines
• Experience with real-time and streaming analytics frameworks (e.g., Spark Structured Streaming, Event Hubs, Kafka)
• Knowledge of enterprise data governance, security masking, access control policies, and data quality frameworks
• Experience building reusable automation frameworks, Docker containers, or Kubernetes orchestration workflows
• Strong self-starter with a proven track record of managing technical deliverables independently and taking end-to-end ownership
• 4+ years of experience writing complex SQL / PL-SQL queries and data transformations
• 2+ years of experience working with Databricks Delta Live Tables (DLT)

*All Telecommuters will be required to adhere to UnitedHealth Group’s Telecommuter Policy.

Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you’ll find a far-reaching choice of benefits and incentives. The salary for this role will range from $91,700 to $163,700 annually based on full-time employment. We comply with all minimum wage laws as applicable.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone–of every race, gender, sexuality, age, location, and income–deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups, and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes — an enterprise priority reflected in our mission.

Diversity creates a healthier atmosphere: UnitedHealth Group is an Equal Employment Opportunity/Affirmative Action employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, disability status, sexual orientation, gender identity or expression, marital status, genetic information, or any other characteristic protected by law.

UnitedHealth Group is a drug-free workplace. Candidates are required to pass a drug test before beginning employment.

UnitedHealth Group is committed to working with and providing reasonable accommodations to individuals with physical and mental disabilities. If you need special assistance or accommodation for any part of the application process, please call 1-866-566-8715 to be connected to Recruitment Services. Recruitment Services hours of operation are 7 a.m. to 7 p.m. CT, Monday through Friday.

UnitedHealth Group is a registered service mark of UnitedHealth Group, Inc. The UnitedHealth Group name with the dimensional logo, as well as the dimensional logo alone, are both service marks for the UnitedHealth Group, Inc.

Diversity creates a healthier atmosphere: UnitedHealth Group is an Equal Employment Opportunity/Affirmative Action employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, disability status, sexual orientation, gender identity or expression, marital status, genetic information, or any other characteristic protected by law.

UnitedHealth Group is a drug-free workplace. Candidates are required to pass a drug test before beginning employment.