Data Engineering Lead

Optum; Dadri; Sur place; INR 3 500 000 - 6 000 000; Plein temps

Résumé du poste

UnitedHealth Group is seeking a Data Engineering Lead to design and optimize scalable data models and pipelines across enterprise data platforms. The role emphasizes Databricks Lakehouse, Bronze-Silver-Gold data layers, and semantic modeling to enable trusted analytics and AI-ready datasets.

You will mentor engineers and drive data governance and CI/CD practices. The ideal candidate brings 7+ years in data engineering, expertise in Spark/PySpark, SQL, and cloud platforms, and strong stakeholder

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, Analytics, or a related field
• 7+ years of experience in Data Engineering, Analytics Engineering, Business Intelligence, or related disciplines
• 3+ years of hands‑on experience with Databricks and Delta Lake technologies
• Hands‑on experience with Power BI, Tableau, or similar BI platforms
• Experience building enterprise‑scale ETL/ELT pipelines
• Experience implementing semantic layers and enterprise reporting solutions
• Experience with performance tuning of Spark workloads and large‑scale analytics environments
• Experience with cloud platforms such as Azure, AWS, or Google Cloud
• Knowledge of CI/CD, DevOps practices, and version control systems such as Git
• Solid understanding of Databricks Lakehouse architecture and Medallion design patterns
• Solid understanding of data governance, metadata management, and data quality frameworks
• Solid expertise in dimensional data modeling, including Star and Snowflake schema design
• Advanced proficiency in SQL and Spark/PySpark development
• Proven excellent problem‑solving, communication, and stakeholder management skills

Responsabilités

• Design, develop, and maintain scalable dimensional data models, including Star and Snowflake schemas
• Build curated data products that support reporting, analytics, AI, and machine learning use cases
• Define and standardize business logic, KPIs, metrics, and calculations across enterprise domains
• Create reusable datasets, semantic models, and data assets to improve consistency and reduce duplication
• Collaborate with business stakeholders to translate requirements into analytical and reporting solutions
• Design and implement modern data architectures leveraging Databricks Lakehouse principles
• Develop and maintain Bronze, Silver, and Gold data layers
• Build scalable ETL/ELT pipelines using Spark, PySpark, SQL, and Databricks Workflows
• Optimize Delta Lake implementations using partitioning, Z-Ordering, data skipping, Change Data Feed
• Implement CI/CD pipelines and DevOps best practices for data engineering and analytics solutions
• Context Layer & Semantic Layer Development
• Design and manage enterprise semantic and context layers to provide a single source of truth for reporting and analytics
• Standardize business metrics, dimensions, hierarchies, and definitions across reporting platforms
• Enable self-service analytics through Power BI, Tableau, and AI-driven applications
• Manage semantic models and governance processes to ensure reporting consistency
• Establish metric certification, lineage tracking, and data governance standards
• Performance Optimization & Cost Management: Analyze query execution plans, optimize Spark workloads, tune Delta Lake tables, monitor usage
• Data Governance & Quality Management: data quality frameworks, metadata management, data cataloging, lineage initiatives
• Technical Leadership & Collaboration: mentor teams, architecture reviews, enterprise data standards, roadmap discussions
• Comply with all applicable Company policies, procedures, and business directives

Connaissances

Databricks

Delta Lake

Power BI

Tableau

SQL

Spark

PySpark

AI/ML data workflows

Cloud platforms

Git

Formation

Bachelor's degree in Computer Science

Master's degree in Computer Science / Data Science

Outils

Databricks

Unity Catalog

Purview/Collibra/Alation

Git

Description du poste

Improve the lives of others while Caring. Connecting. Growing together.

Job Description - Data Engineering Lead (2387751)

Data Engineering Lead - 2387751

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities:

• Design, develop, and maintain scalable dimensional data models, including Star and Snowflake schemas
• Build curated data products that support reporting, analytics, AI, and machine learning use cases
• Define and standardize business logic, KPIs, metrics, and calculations across enterprise domains
• Create reusable datasets, semantic models, and data assets to improve consistency and reduce duplication
• Collaborate with business stakeholders to translate requirements into analytical and reporting solutions
• Design and implement modern data architectures leveraging Databricks Lakehouse principles
• Develop and maintain Bronze, Silver, and Gold data layers
• Build scalable ETL/ELT pipelines using Spark, PySpark, SQL, and Databricks Workflows
• Optimize Delta Lake implementations using partitioning, Z-Ordering, data skipping, Change Data Feed (CDF)
• Implement CI/CD pipelines and DevOps best practices for data engineering and analytics solutions
• Context Layer & Semantic Layer Development
• Design and manage enterprise semantic and context layers to provide a single source of truth for reporting and analytics
• Standardize business metrics, dimensions, hierarchies, and definitions across reporting platforms
• Enable self-service analytics through Power BI, Tableau, and AI-driven applications
• Manage semantic models and governance processes to ensure reporting consistency
• Establish metric certification, lineage tracking, and data governance standards
• Performance Optimization & Cost Management
  • Analyze query execution plans and identify performance bottlenecks
  • Optimize Spark workloads for scalability, reliability, and cost efficiency
  • Implement cluster sizing, autoscaling, caching, broadcast joins, and Adaptive Query Execution (AQE) strategies
  • Tune Delta Lake tables through file compaction, partition optimization, and storage management
  • Monitor warehouse and cluster utilization to improve performance and control cloud expenses
  • Enhance dashboard and reporting performance through optimized data models and query design
• Data Governance & Quality Management
  • Develop and implement data quality frameworks, validation processes, and automated monitoring
  • Ensure compliance with enterprise security, governance, and regulatory requirements
  • Support metadata management, data cataloging, and lineage initiatives
  • Establish monitoring and alerting for data pipelines, processing failures, and data anomalies
  • Partner with governance teams to drive data stewardship and certification processes
• Technical Leadership & Collaboration
  • Collaborate with data scientists, analysts, engineers, product owners, and business leaders
  • Mentor and guide junior analytics engineers and data engineers
  • Conduct architecture reviews and recommend technology best practices
  • Drive adoption of enterprise data standards, frameworks, and governance processes
  • Lead technical design discussions and influence data platform strategy and roadmap
• Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications:

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, Analytics, or a related field
• 7+ years of experience in Data Engineering, Analytics Engineering, Business Intelligence, or related disciplines
• 3+ years of hands‑on experience with Databricks and Delta Lake technologies
• Hands‑on experience with Power BI, Tableau, or similar BI platforms
• Experience building enterprise‑scale ETL/ELT pipelines
• Experience implementing semantic layers and enterprise reporting solutions
• Experience with performance tuning of Spark workloads and large‑scale analytics environments
• Experience with cloud platforms such as Azure, AWS, or Google Cloud
• Knowledge of CI/CD, DevOps practices, and version control systems such as Git
• Solid understanding of Databricks Lakehouse architecture and Medallion (Bronze/Silver/Gold) design patterns
• Solid understanding of data governance, metadata management, and data quality frameworks
• Solid expertise in dimensional data modeling, including Star and Snowflake schema design
• Advanced proficiency in SQL and Spark/PySpark development
• Proven excellent problem‑solving, communication, and stakeholder management skills

Preferred Qualifications:

• Master's degree in Computer Science, Data Science, Analytics, or a related discipline
• Databricks certifications (Data Engineer Associate, Professional, or equivalent)
• Experience with Databricks Unity Catalog and enterprise data governance implementations
• Experience supporting AI/ML, Generative AI, or advanced analytics use cases
• Experience designing enterprise semantic models and metric layers
• Healthcare, insurance, or highly regulated industry experience
• Demonstrated experience leading technical teams, mentoring engineers, and driving enterprise data strategy
• Knowledge of data catalog and governance tools such as Purview, Collibra, or Alation
• Spark/PySpark Development
• Data Governance & Quality
• Performance Optimization
• Semantic Layer Design
• Technical Leadership
• Stakeholder Management

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone–of every race, gender, sexuality, age, location and income–deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes — an enterprise priority reflected in our mission.

UnitedHealth Group is committed to working with and providing reasonable accommodations to individuals with physical and mental disabilities. If you need special assistance or accommodation for any part of the application process, please call 1-866-566-8715 to be connected to Recruitment Services. Recruitment Services hours of operation are 7 a.m. to 7 p.m. CT, Monday through Friday.

UnitedHealth Group is a registered service mark of UnitedHealth Group, Inc. The UnitedHealth Group name with the dimensional logo, as well as the dimensional logo alone, are both service marks for the UnitedHealth Group, Inc.

Diversity creates a healthier atmosphere: UnitedHealth Group is an Equal Employment Opportunity/Affirmative Action employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, disability status, sexual orientation, gender identity or expression, marital status, genetic information, or any other characteristic protected by law.

UnitedHealth Group is a drug‑free workplace. Candidates are required to pass a drug test before beginning employment.