Software Engineer - Data Engineer

Mployee.me; Hyderabad; On-site; INR 1,000,000 - 1,400,000; Full time

Job summary

Optum, a global health services company, is seeking a data engineer to design scalable data pipelines and contribute to analytics platforms in Hyderabad.

You will work on Azure Databricks, Spark-based ETL/ELT processes, and ML-ready data products, collaborating with cross-functional teams to accelerate health insights. A strong background in SQL, Python, and cloud data services is required, with mentoring responsibilities and best-practice adherence.

Qualifications

• 3+ years in data engineering, data integration, data modeling, and ETL/ELT.
• 3+ years in Python.
• 3+ years in API design and lifecycle management (GraphQL, REST).
• 2+ years Redis-backed state management and event-driven processing.
• 2+ years Apache Spark (PySpark/Spark SQL).
• 2+ years Azure Databricks with UC, Snowflake, Functions, Service Bus.
• 2+ years SQL with complex schemas and query optimization.
• Experience building LLM integrations for automation.
• Experience with Terraform and CI/CD (GitHub Actions, Git, Artifactory, Sonar).

Responsibilities

• Design and develop ETL/ELT on Azure Databricks and Spark.
• Build large-scale data pipelines for ML, BI, and analytics.
• Optimize ETL/ELT pipelines for analytics and ML features.
• Ingest data from RDBMS, APIs, FTPs and cloud storage; multiple formats.
• Write advanced SQL with performance tuning.
• Develop AI-based solutions to automate tasks and improve efficiency.
• Improve data reliability, integrity, and system quality.
• Shape data engineering architectures and standards.
• Mentor data engineers on cloud data platforms.
• Follow company policies and flexible work arrangements.

Skills

Python

API design

Redis state management

Apache Spark

Azure Databricks

SQL

LLM integrations

DevOps CI/CD

Tools

UC/Snowflake/Functions/Service Bus

Job description

Job Summary

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

• Design and develop ETL/ELT solutions on Azure Databricks, LakeBase and Spark
• Develop, implement, and deploy large scale data pipelines empowering machine learning algorithms, insights generation, business intelligence dashboards, reporting and new data products
• Design, build, optimize, and manage modern large-scale data pipelines ETL/ELT processing to support data integration for analytics, machine learning features and predictive modelling
• Consume data from a variety of sources (RDBMS, APIs, FTPs and other cloud storage) & formats (Excel, CSV, XML, JSON, Parquet, Unstructured)
• Write advanced / complex SQL with performance tuning and optimization
• Build AI based solutions for solving business needs, process automations and improving operational efficiency
• Identify ways to improve data reliability, data integrity, system efficiency and quality
• Participate in architectural evolution of data engineering patterns, frameworks, systems, and platforms including defining best practices and standards for managing data collections and integration
• Mentor other data engineers and provide technical direction by teaching other data engineers how to leverage cloud data platforms
• Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications

• 3+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes to provide quality data and analytics solutions
• 3+ years of experience in Python
• 3+ years of experience with API design and lifecycle management (GraphQL, REST, etc.)
• 2+ years of hands‑on experience with Redis-backed state management and event‑driven processing
• 2+ years of experience in Apache Spark (PySpark/Spark SQL)
• 2+ years of experience building and deploying cloud-based solutions using Azure Databricks with UC, Snowflake, Functions, Service Bus
• 2+ years of experience in SQL with designing complex data schemas and query performance optimization
• Experience building LLM integrations for workflow automation/business needs
• Experience with DevOps automation with Terraform and CI/CD processes/tools (GitHub Actions, Git, Artifactory, Sonar)