Data Engineering Consultant (Python,Pyspark OR Scala, Data Bricks)

UnitedHealth Group; Dadri; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

UnitedHealth Group in India is seeking a senior Big Data engineer to design, code, test, and maintain scalable data applications using PySpark and Scala on Azure. You will build analytics rules engines and predictive algorithms, and contribute to a robust data platform for batch and streaming workloads.

The role requires strong SQL/PLSQL, CI/CD, Airflow, and experience with Kubernetes/Azure services. You will collaborate across teams, document work, and support production systems while pursuing

Qualifications

• Bachelor or Master in computer science or information technology or equivalent.
• 7+ years of work experience in big data / analytics.
• Proficient in Python, PySpark, and Scala.
• Experience with Airflow, SQL/PLSQL, and cloud environments (Azure).
• CI/CD experience with Jenkins or GitHub Actions.
• Knowledge of Spark, Hadoop, and data lake technologies.

Responsibilities

• Design, code, test, document, and maintain scalable Big Data applications using PySpark/Scala on Azure.
• Develop analytics rules engines and predictive algorithms with unit testing (pytest).
• Deploy automation for batch and streaming platforms using Kafka; optimize performance.
• Schedule pipelines with Airflow and ensure proper job dependencies.
• Write technical documentation and provide production support.
• Collaborate across teams to drive solution-focused outcomes.
• Design and deploy AI-powered no-code/low-code solutions to meet business needs.

Skills

Python

PySpark

Scala

Airflow

Azure Cloud

SQL/PLSQL

Shell scripting

CI/CD (Jenkins, GitHub Actions)

Agile

Big Data stack (Spark/Hadoop)

Education

Bachelor's / Master's in CS or IT

Tools

Azure Data Lake

Azure Databricks

Azure Kubernetes

Azure Service Bus

Azure Key Vault

Azure Container Registry

Job description

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities:

• Design, Code, Test, document, and maintain high-quality and scalable Big Data, application using PySpark and Scala Spark on Cloud solutions on Azure
• Develop and implement analytics rules engines and Predictive Algorithm and Unit Testing framework eg PyTest
• Research, evaluate, and deploy new automation tools, frameworks, and patterns to build sustainable Big Data platform for batch and streaming applications using Kafka
• Identify gaps and opportunities for improvement of existing solutions
• Ensure the code is meeting the desired quality checks using Sonar
• Schedule the deployed pipelines using Airflow following proper dependency hierarchy of jobs
• Write technical documentation and participate in production support
• Performance Optimization and application migration experience
• Builder Responsibilities:
  • Design, develop, and deploy AI-powered solutions using no-code, low-code, and advanced platforms, translating business needs into scalable applications that enhance products, workflows, and decision-making
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• Bachelor/master's degree in computer science or information technology or equivalent with 7+ years of work experience
• Solid experience in programming languages like Python,Pyspark, Scala
• Good experience in job schedulers like Airflow
• Good experience in working on cloud environment, preferably Azure
• Proven experience in cloud services like Azure Kubernetes, Azure Data Lake, Azure Databricks, Azure Container Registries, Azure Service Bus, Azure Key Vault
• Solid experience and expertise in writing complex queries in SQL and PL/SQL
• Experience in writing Shell scripting and Snowflake
• Experience with Continuous Integration CI/CD pipelines using Jenkins, GitHub Actions
• Experience working in an agile environment
• Experience working across teams and proven track record in solution focused problem-solving skills
• Expertise in BigData frameworks like Spark and good understanding of Hadoop concepts

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.