Data Engineer - Pyspark, Databricks, Snowflake, Azure Cloud

Optum India; Hyderabad; On-site; INR 1,500,000 - 2,700,000; Full time

Job summary

Optum India is seeking a senior Data Engineer to design and operate scalable ETL/ELT pipelines across diverse data sources in a cloud-native environment. You will develop data solutions on Databricks and Snowflake, ensuring high data quality and governance while enabling analytics and AI/ML workloads.

With 5+ years in data engineering, you will collaborate with product, analytics, and business teams to translate requirements into production-ready data platforms, applying CI/CD and DevOps

Qualifications

• Bachelor's degree in Computer Science or related field.
• 5+ years of experience as a Data Engineer or in data platform / analytics engineering roles.
• 3+ years of experience with SQL, data modeling, and data transformations.
• 3+ years of hands-on experience with Databricks and/or Snowflake and large-scale data processing.
• 3+ years of experience building ETL/ELT pipelines, data warehouses, and analytical data stores.
• 3+ years of experience with source control and DevOps practices (Azure DevOps, GitHub, CI/CD).

Responsibilities

• Design, build, and operate scalable ETL/ELT pipelines for structured and semi-structured data across internal and external sources.
• Develop cloud-native data solutions on modern data platforms (e.g., Databricks lakehouse, Snowflake).
• Apply data modeling, transformations, and enrichment to produce trusted, analytics-ready datasets.
• Ensure data quality, reliability, and auditability through validation, reconciliation, and monitoring.
• Implement and follow enterprise data governance, security, and compliance standards.
• Optimize data workloads for performance, cost efficiency, and operational reliability.
• Support modernization efforts, including migration of legacy ETL pipelines to cloud-native architectures.
• Partner with product, analytics, and business stakeholders to translate requirements into production-grade data solutions.
• Enable downstream analytics, reporting, and AI/ML workloads through well-designed data assets.
• Apply software engineering best practices, including CI/CD, version control, testing, and code reviews.
• Contribute to architectural discussions and mentor junior engineers.
• Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI.

Skills

SQL

ETL/ELT pipelines

Databricks

Snowflake

Data modeling

CI/CD

GitHub

Azure DevOps

Data warehouses

Analytics engineering

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Snowflake

Azure DevOps

GitHub

CI/CD

Job description

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

• Design, build, and operate scalable ETL/ELT pipelines for structured and semi‑structured data across internal and external sources
• Develop cloud‑native data solutions on modern data platforms (e.g., Databricks lakehouse, Snowflake)
• Apply data modeling, transformations, and enrichment to produce trusted, analytics‑ready datasets
• Ensure data quality, reliability, and auditability through validation, reconciliation, and monitoring
• Implement and follow enterprise data governance, security, and compliance standards
• Optimize data workloads for performance, cost efficiency, and operational reliability
• Support modernization efforts, including migration of legacy ETL pipelines to cloud‑native architectures
• Partner with product, analytics, and business stakeholders to translate requirements into production‑grade data solutions
• Enable downstream analytics, reporting, and AI/ML workloads through well‑designed data assets
• Apply software engineering best practices, including CI/CD, version control, testing, and code reviews
• Contribute to architectural discussions and mentor junior engineers
• Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re‑assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications

• Bachelor's degree in Computer Science, Engineering related field
• 5+ years of experience as a Data Engineer or in data platform / analytics engineering roles
• 3+ years of experience with SQL, data modeling, and data transformations
• 3+ years of hands‑on experience with Databricks and/or Snowflake and large‑scale data processing
• 3+ years of experience building ETL/ELT pipelines, data warehouses, and analytical data stores
• 3+ years of experience with source control and DevOps practices (Azure DevOps, GitHub, CI/CD)

Preferred Qualifications

• Experience building enterprise‑scale data platforms
• Experience with real‑time/streaming frameworks (Structured Streaming)
• Exposure to GenAI tools (e.g., GitHub Copilot) for engineering productivity
• Solid understanding of secure coding practices and vulnerability remediation
• Proven ability to work in distributed, cross‑functional global teams
• Proven ability to analyze logs, troubleshoot production issues, and optimize performance
• Proven solid skills in performance tuning and debugging large‑scale pipelines
• Demonstrated capability to design and deploy AI-powered solutions responsibly