Software Engineer - Data Engineer

Optum; Hyderabad; On-site; INR 900,000 - 1,600,000; Full time

Job summary

Optum in Hyderabad, India is seeking a data engineer to design and implement scalable ETL/ELT pipelines on Azure Databricks, empower ML‑driven analytics and BI dashboards, and integrate data from diverse sources.

You will optimize SQL, build data models, and contribute to architecture, while mentoring peers and aligning with cloud data platform standards.

Qualifications

• 3+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes to provide quality data and analytics solutions
• 3+ years of experience in Python
• 3+ years of experience with API design and lifecycle management (GraphQL, REST, etc.)
• 2+ years of hands‑on experience with Redis-backed state management and event‑driven processing
• 2+ years of experience in Apache Spark (PySpark/Spark SQL)
• 2+ years of experience building and deploying cloud-based solutions using Azure Databricks with UC, Snowflake, Functions, Service Bus
• 2+ years of experience in SQL with designing complex data schemas and query performance optimization
• Experience building LLM integrations for workflow automation/business needs
• Experience with DevOps automation with Terraform and CI/CD processes/tools (GitHub Actions, Git, Artifactory, Sonar)

Responsibilities

• Design and develop ETL/ELT solutions on Azure Databricks, LakeBase and Spark
• Develop, implement, and deploy large scale data pipelines empowering ML algorithms, insights generation, BI dashboards, reporting and new data products
• Design, build, optimize, and manage modern large-scale data pipelines ETL/ELT processing for analytics, ML features and predictive modelling
• Consume data from RDBMS, APIs, FTPs and other cloud storage and formats (Excel, CSV, XML, JSON, Parquet, Unstructured)
• Write advanced SQL with performance tuning and optimization
• Build AI based solutions for workflow automation and operational efficiency
• Identify ways to improve data reliability, data integrity, system efficiency and quality
• Participate in architectural evolution of data engineering patterns and standards
• Mentor other data engineers and provide technical direction on cloud data platforms
• Comply with company policies and work location / schedules alignment

Skills

ETL/ELT design

Python

Spark (PySpark)

SQL optimization

API design & lifecycle

Azure Databricks

Machine learning data pipelines

Data modeling

Mentoring teammates

Tools

Azure Databricks

Snowflake

Azure Functions

Service Bus

GitHub Actions

Terraform

Artifactory

SonarQube

Job description

Job Summary

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

• Design and develop ETL/ELT solutions on Azure Databricks, LakeBase and Spark
• Develop, implement, and deploy large scale data pipelines empowering machine learning algorithms, insights generation, business intelligence dashboards, reporting and new data products
• Design, build, optimize, and manage modern large-scale data pipelines ETL/ELT processing to support data integration for analytics, machine learning features and predictive modelling
• Consume data from a variety of sources (RDBMS, APIs, FTPs and other cloud storage) & formats (Excel, CSV, XML, JSON, Parquet, Unstructured)
• Write advanced / complex SQL with performance tuning and optimization
• Build AI based solutions for solving business needs, process automations and improving operational efficiency
• Identify ways to improve data reliability, data integrity, system efficiency and quality
• Participate in architectural evolution of data engineering patterns, frameworks, systems, and platforms including defining best practices and standards for managing data collections and integration
• Mentor other data engineers and provide technical direction by teaching other data engineers how to leverage cloud data platforms
• Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications

• 3+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes to provide quality data and analytics solutions
• 3+ years of experience in Python
• 3+ years of experience with API design and lifecycle management (GraphQL, REST, etc.)
• 2+ years of hands‑on experience with Redis-backed state management and event‑driven processing
• 2+ years of experience in Apache Spark (PySpark/Spark SQL)
• 2+ years of experience building and deploying cloud-based solutions using Azure Databricks with UC, Snowflake, Functions, Service Bus
• 2+ years of experience in SQL with designing complex data schemas and query performance optimization
• Experience building LLM integrations for workflow automation/business needs
• Experience with DevOps automation with Terraform and CI/CD processes/tools (GitHub Actions, Git, Artifactory, Sonar)