Sr Data Engineering Analyst-Python, Scala, Azure

Optum India; Dadri; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Optum India is seeking a Data Engineer (Grade 26) to design, develop, and maintain scalable Big Data solutions on Azure Cloud. You will build reliable data pipelines using PySpark, Python, and SQL, while collaborating with senior engineers and cross-functional teams.

You will work with Airflow for orchestration, ensure secure data operations with Azure Key Vault, and contribute to production support and documentation. A growth path into advanced cloud-native architectures is offered.

Qualifications

• Bachelor's or master's degree in Computer Science, Information Technology, or equivalent.
• Hands-on Python for data engineering and application development.
• Strong Spark knowledge and Hadoop ecosystem understanding.
• Experience with Airflow and job scheduling.
• Cloud experience, preferably Azure, with Azure data services.

Responsibilities

• Design, code, test, and maintain scalable data applications using Python, PySpark and Scala Spark on Azure.
• Develop data pipelines and schedule workflows with Apache Airflow.

Skills

Python

Apache Spark

SQL/PLSQL

Airflow

Azure

Shell scripting

CI/CD

Prompt engineering

Teamwork

Analytical skills

Education

Bachelor's or Master's in CS/IT

Tools

Azure Data Lake

Azure Databricks

Azure Data Factory

Azure Key Vault

Snowflake

GitHub Actions

Jenkins

FastAPI

Job description

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

The Data Engineer (Grade 26) will support the design, development, and maintenance of scalable Big Data solutions on cloud platforms. This role is ideal for data engineers with hands-on experience in Spark-based data processing, Azure cloud services, real-time data processing and data pipeline orchestration.

The individual will work closely with senior engineers, architects, and cross-functional teams to build reliable data pipelines, improve existing solutions, and ensure secure and efficient data operations. The role emphasizes solid fundamentals in data engineering, cloud technologies, and production support, while providing opportunities to grow into advanced Big Data and cloud-native architectures.

Primary Responsibilities

• Design, code, test, document, and maintain high-quality, scalable data applications using Python, PySpark and Scala Spark on Azure Cloud platforms
• Develop and manage data pipelines and schedule workflows using Apache Airflow, ensuring proper job dependencies and execution order
• Securely manage secrets and credentials using Azure Key Vault, following enterprise security best practices
• Analyze existing data pipelines and applications to identify gaps, risks, and opportunities for improvement
• Assist in analyzing data architecture and design frameworks, working with multiple databases and data warehouses
• Create prototypes and proof-of-concepts (POCs) and participate in design and code reviews
• Write and maintain technical documentation for data pipelines, workflows, and operational procedures
• Participate in production support activities, including monitoring, troubleshooting, and issue resolution
• Perform performance optimization of data pipelines and assist with application migration efforts across environments
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or equivalent, with more than 4 years of relevant work experience
• Hands-on experience with Python for data engineering and application development
• Working experience with Apache Spark and good understanding of Hadoop ecosystem concepts
• Experience with job scheduling and orchestration tools, particularly Apache Airflow
• Good experience working in cloud environments, preferably Microsoft Azure
• Hands-on exposure to Azure Data Lake, Azure Databricks, Azure Data Factory, and Azure Key Vault
• Solid experience in writing complex SQL and PL/SQL queries
• Experience with Snowflake and writing Shell scripts for automation and operational tasks
• Exposure to CI/CD pipelines using tools such as Jenkins and GitHub Actions
• Hands-on exposure to AI-assisted development tools such as Microsoft Copilot, with a basic understanding of prompt engineering to improve coding efficiency, data analysis productivity, and documentation quality
• Basic understanding of software development best practices, version control, and collaborative development
• Proven good analytical skills, attention to detail, and willingness to learn new technologies and frameworks
• Demonstrated ability to work effectively in a team-oriented and fast-paced environment

Preferred Qualification

• Experience in developing applications using FastAPI

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.