Data Engineering Consultant (Python,Pyspark OR Scala, Data Bricks)

Optum India; Dadri; Presencial; INR 2 800 000 - 5 000 000; Tempo integral

Resumo da oferta

Optum India is seeking an experienced Big Data Engineer to design, code, and optimize scalable data applications using PySpark and Scala on Azure. You will build analytics rules engines, predictive algorithms, and support batch and streaming pipelines with Kafka and Airflow.

You will collaborate across teams, write technical documentation, and participate in production support, emphasizing code quality with Sonar and CI/CD practices in a fast-paced cloud environment.

Qualificações

• Bachelor's or Master's degree in computer science or IT with 7+ years of experience.
• Strong coding experience in Python, PySpark, and Scala.
• Hands-on with Airflow and Azure cloud services.
• Expertise in SQL/PL-SQL and data warehousing tools.

Responsabilidades

• Design, code, test, and maintain scalable Big Data applications using PySpark/Scala on Azure.
• Develop analytics rules engines and predictive algorithms; implement unit tests.
• Deploy batch and streaming pipelines with Kafka and Airflow; ensure quality via Sonar.
• Prepare technical documentation and provide production support.
• Lead AI-powered builder solutions using no-code/low-code platforms.

Conhecimentos

Python

PySpark

Scala

Airflow

Azure

SQL/PL-SQL

Shell scripting

Snowflake

Jenkins

GitHub Actions

CI/CD

Hadoop

Kubernetes

Big Data

Spark

Formação académica

Bachelor's or Master's in Computer Science / IT

Ferramentas

Azure Kubernetes

Azure Data Lake

Azure Databricks

Azure Container Registries

Azure Service Bus

Azure Key Vault

Descrição da oferta de emprego

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

• Design, Code, Test, document, and maintain high-quality and scalable Big Data, application using PySpark and Scala Spark on Cloud solutions on Azure
• Develop and implement analytics rules engines and Predictive Algorithm and Unit Testing framework eg PyTest
• Research, evaluate, and deploy new automation tools, frameworks, and patterns to build sustainable Big Data platform for batch and streaming applications using Kafka
• Identify gaps and opportunities for improvement of existing solutions
• Ensure the code is meeting the desired quality checks using Sonar
• Schedule the deployed pipelines using Airflow following proper dependency hierarchy of jobs
• Write technical documentation and participate in production support
• Performance Optimization and application migration experience
• Builder Responsibilities:
  • Design, develop, and deploy AI-powered solutions using no-code, low-code, and advanced platforms, translating business needs into scalable applications that enhance products, workflows, and decision-making
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
  

Required Qualifications

• Bachelor/master's degree in computer science or information technology or equivalent with 7+ years of work experience
• Solid experience in programming languages like Python,Pyspark, Scala
• Good experience in job schedulers like Airflow
• Good experience in working on cloud environment, preferably Azure
• Proven experience in cloud services like Azure Kubernetes, Azure Data Lake, Azure Databricks, Azure Container Registries, Azure Service Bus, Azure Key Vault
• Solid experience and expertise in writing complex queries in SQL and PL/SQL
• Experience in writing Shell scripting and Snowflake
• Experience with Continuous Integration CI/CD pipelines using Jenkins, GitHub Actions
• Experience working in an agile environment
• Experience working across teams and proven track record in solution focused problem-solving skills
• Expertise in BigData frameworks like Spark and good understanding of Hadoop concepts

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.