Senior Data Engineering Lead

Optum India; Hyderabad; On-site; INR 4,500,000 - 6,500,000; Full time

Job summary

Optum India is seeking an experienced data engineering professional to join our team in Hyderabad. The role focuses on building scalable data solutions using Azure Databricks, Spark, and Scala, with strong emphasis on cloud deployment, DL/ETL pipelines, and API development.

You will collaborate across teams to optimize performance, manage data warehousing tasks, and implement AI-assisted tooling to enhance productivity and quality of deliverables.

Qualifications

• 10+ years of proven experience with Azure Databricks, Apache Spark, and Scala.
• 5+ years of experience with Microsoft Azure.
• Experience with data warehousing solutions and ETL tools.
• Proven excellent problem-solving skills and attention to detail.
• Proven solid communication and collaboration skills.
• Solid experience in working with Databricks, including its architecture, features, and capabilities.
• Solid experience in Cloud-based deployment platforms like Azure, Kubernetes, etc.
• Experience with RDBMS(Azure SQL Server).
• DevOps hands on experience like build and deployment pipeline creations.
• API development experience like REST or GraphQL.
• In-depth knowledge and hands-on experience with Scala programming language.
• Expertise in Apache Spark, including Spark RDDs, DataFrames, and Spark SQL.
• Knowledge on Java / J2EE or .Net.
• Solid understanding of distributed computing principles and big data processing.
• Familiarity with big data technologies such as Hadoop, Hive, HDFS and data lakes.
• Proficiency in writing complex SQL queries and working with relational databases.
• Demonstrated ability to adopt AI tools in application like Github Copilot, M365 Copilot, StoryCraft, GenAI etc.
• Demonstrated ability to adopt AI tools in applications like Github Copilot, StoryCraft, GenAI etc.

Responsibilities

• Comply with the terms and conditions of the employment contract, company policies and procedures, and any directives (such as transfers or re-assignments, changes in teams or work shifts, flexible benefits, alternate work arrangements).

Skills

Azure Databricks

Apache Spark

Scala

Microsoft Azure

ETL tools

problem solving

communication

Databricks architecture

Cloud deployment

RDBMS Azure SQL

DevOps pipelines

REST/GraphQL API

Java/J2EE/.Net

Hadoop/Hive/HDFS

SQL queries

AI tool adoption

Tools

Databricks

Azure

Kubernetes

GitHub Copilot

M365 Copilot

StoryCraft

Job description

Job Description:

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start

Caring. Connecting. Growing together.

Primary Responsibility

• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications

• 10+ years of proven experience with Azure Databricks, Apache Spark, and Scala
• 5+ years of experience with Microsoft Azure
• Experience with data warehousing solutions and ETL tools
• Proven excellent problem-solving skills and attention to detail
• Proven solid communication and collaboration skills
• Solid experience in working with Databricks, including its architecture, features, and capabilities
• Solid experience in Cloud-based deployment platforms like Azure, Kubernetes, etc.
• Experience with RDBMS(Azure SQL Server)
• DevOps hands on experience like build and deployment pipeline creations
• API development experience like REST or GraphQL
• In-depth knowledge and hands-on experience with Scala programming language, including its functional programming concepts, collections, and advanced language features.
• Expertise in Apache Spark, including Spark RDDs, DataFrames, and Spark SQL. Understanding of Spark's distributed computing model and ability to optimize and tune Spark jobs for performance
• Knowledge on Java / J2EE or .Net
• Solid understanding of distributed computing principles and big data processing
• Familiarity with big data technologies and concepts, such as Hadoop, Hive, HDFS, and data lakes. Experience in working with large-scale datasets and processing them efficiently
• Proficiency in writing complex SQL queries and working with relational databases
• Demonstrated ability to adopt AI tools in application like Github Copilot, M365 Copilot, StoryCraft, GenAI etc.
• Demonstrated ability to adopt AI tools in application like Github Copilot, StoryCraft, GenAI etc

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.