Senior Data Engineering Lead

Optum; Hyderabad; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Optum is seeking a senior data engineer to drive big data initiatives on Azure, with hands-on work on Databricks, Spark, and Scala. You will design and optimize data pipelines, collaborating with cross-functional teams to deliver scalable solutions for healthcare analytics.

The role requires strong Python/Scala coding, experience with distributed systems, and the ability to leverage AI-assisted tools to improve productivity and outcomes.

Qualifications

• 10+ years of proven experience with Azure Databricks, Apache Spark, and Scala.
• 5+ years of experience with Microsoft Azure.
• Experience with data warehousing solutions and ETL tools.
• Proven problem-solving, communication, and collaboration skills.

Responsibilities

• Develop and optimize large-scale data processing pipelines using Spark, Databricks, and Azure.

Skills

Azure Databricks

Apache Spark

Scala

Microsoft Azure

Data Warehousing

ETL Tools

API Development

Kubernetes

SQL (Azure SQL Server)

Big Data

Tools

Kubernetes

Job description

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibility:

• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• 10+ years of proven experience with Azure Databricks, Apache Spark, and Scala
• 5+ years of experience with Microsoft Azure
• Experience with data warehousing solutions and ETL tools
• *Proven excellent problem-solving skills and attention to detail
• Proven solid communication and collaboration skills
• Solid experience in working with Databricks, including its architecture, features, and capabilities
• Solid experience in Cloud-based deployment platforms like Azure, Kubernetes, etc.
• Experience with RDBMS(Azure SQL Server)
• DevOps hands on experience like build and deployment pipeline creations
• API development experience like REST or GraphQL
• In-depth knowledge and hands-on experience with Scala programming language, including its functional programming concepts, collections, and advanced language features.
• Expertise in Apache Spark, including Spark RDDs, DataFrames, and Spark SQL. Understanding of Spark''s distributed computing model and ability to optimize and tune Spark jobs for performance
• Knowledge on Java / J2EE or .Net
• Solid understanding of distributed computing principles and big data processing
• Familiarity with big data technologies and concepts, such as Hadoop, Hive, HDFS, and data lakes. Experience in working with large-scale datasets and processing them efficiently
• Proficiency in writing complex SQL queries and working with relational databases
• Demonstrated ability to adopt AI tools in application like Github Copilot, M365 Copilot, StoryCraft, GenAI etc.
• Demonstrated ability to adopt AI tools in application like Github Copilot, StoryCraft, GenAI etc

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.