Lead Data Engineer

Omnicom Global Solutions; Hyderabad; Hybrid; INR 4,000,000 - 6,500,000; Full time

Job summary

Omnicom Global Solutions in Hyderabad is seeking a Lead Engineer for Databricks and Data Engineering to drive end-to-end data pipelines, mentor engineers, and partner with onshore and offshore teams in an agile environment.

You will design and implement Bronze-to-Gold Databricks pipelines, ensure data quality and security, and help govern data warehouses for ERP and financial data such as JDE.

Qualifications

• 7+ to 8+ years of data engineering experience with hands-on Databricks
• Strong SQL and Python skills for code reviews
• Experience building pipelines from ERP/financial source systems (JD Edwards/JDE) to governed data warehouses
• Databricks Certified Data Engineer Professional (Associate acceptable if in progress)

Responsibilities

• Lead daily check-ins with offshore pod and translate backlog into sprint tasks
• Design and build Databricks pipelines (Bronze through Gold) for financial and operational data
• Own code reviews, data quality validation, and enforce platform security (Unity Catalog)
• Lead knowledge transfer on Boomi integration patterns (JDE-Concur, JDE-UKG) and mentor engineers
• Proactively identify dependencies, clarify requirements, and communicate capacity constraints

Skills

Databricks

Spark/PySpark

Delta Lake

Unity Catalog

Workflows

SQL

Python

Job description

Role Overview

• Position: Lead Engineer, Databricks & Data Engineering
• Company: Omnicom Global Solutions (OGS)
• Experience Required: 7+ years
• Location: Hyderabad,
• Core Summary: A hybrid technical-leadership role acting as the primary liaison between the onshore lead and an offshore engineering pod. You will drive agile execution, build end-to-end data pipelines, mentor engineers, and ensure technical quality without letting ambiguity slow down delivery.

Key Responsibilities

• Agile & Team Leadership: Manage daily check-ins with the offshore pod, translate backlog items into clear sprint tasks, track progress, and run daily alignment meetings with the onshore technical lead.
• Technical Execution: Design and build complex Databricks pipelines (Bronze through Gold layers) for financial and operational data.
• Quality & Security: Own code reviews, data quality validation, and enforce platform security (Row-Level/Column-Level Security via Unity Catalog).
• Integration & Mentorship: Lead knowledge transfer from onshore teams on Boomi integration patterns (e.g., JDE-Concur, JDE-UKG) and mentor senior/data engineers.
• Risk & Scope Management: Proactively identify dependencies, clarify ambiguous requirements, and communicate capacity constraints early.

Required Qualifications

• Experience: 7+ to 8+ years of data engineering experience with heavy hands-on Databricks expertise (Spark/PySpark, Delta Lake, Unity Catalog, Workflows).
• Certifications: Databricks Certified Data Engineer Professional (Associate accepted if Professional is in progress).
• Tech Stack: Strong SQL and Python skills for code reviews.
• Domain Expertise: Experience building pipelines from financial/ERP source systems JD Edwards (JDE) strongly preferred (EcoSys or MS Dynamics accepted)—into governed data warehouses/lakehouses (General Ledger, project P&L, cost data).