Senior Databricks Data Engineer

Omnicom Global Solutions; Bengaluru; Hybrid; INR 1,800,000 - 2,400,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

Omnicom Global Solutions in Bengaluru, India invites a Senior Databricks Data Engineer to lead a strategic migration from Informatica to Azure Databricks, modernizing ingestion, transformation, and analytics within a Lakehouse architecture.

The role requires hands-on Azure Databricks, PySpark, SQL, and Oracle DB expertise, with the ability to deliver end-to-end solutions, collaborate with architects and stakeholders, and mentor junior engineers in a client-facing environment.

Qualifications

• 4–6 years of hands-on experience with Azure Databricks in enterprise environments.
• Proven experience delivering large-scale data migration and modernization projects.
• Strong experience migrating ETL processes from Informatica or similar ETL platforms to cloud-native architectures.
• Strong understanding of Lakehouse architecture and modern data engineering principles.
• Extensive experience in Data Modelling, Data Warehousing, and Data Lake design.
• Experience working with large-scale enterprise data platforms.
• Strong analytical and problem-solving abilities.
• Experience working with Agile delivery methodologies.
• Ability to communicate complex technical concepts to both technical and non-technical stakeholders.
• Experience working in client-facing environments and collaborating with business stakeholders.
• Strong understanding of software development lifecycle and engineering best practices.

Responsibilities

• Lead the analysis, design, development, testing, and deployment of data engineering solutions on Azure Databricks.
• Participate in the migration of existing ETL/ELT workloads from Informatica to Azure Databricks.
• Analyze Informatica mappings, workflows, and dependencies and translate them into scalable Databricks solutions.
• Design and develop performant batch and streaming data pipelines using PySpark and SQL.
• Build and maintain data ingestion frameworks for structured and semi-structured data sources.
• Develop, optimize, and maintain Delta Lake-based data architectures using Medallion Architecture principles (Bronze, Silver, Gold).
• Work closely with Solution Architects and Business Analysts to understand business requirements and translate them into technical solutions.
• Implement data quality, reconciliation, validation, and monitoring processes throughout the migration lifecycle.
• Create technical specifications, design documents, and operational support documentation.
• Participate in code reviews and enforce engineering best practices and development standards.
• Support deployment activities through CI/CD pipelines and Infrastructure-as-Code frameworks.
• Collaborate with stakeholders, project managers, and delivery teams to ensure successful project outcomes.
• Mentor junior engineers and provide technical leadership across the project team.
• Support production releases and troubleshoot complex data issues when required.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

Lakehouse Architecture

Medallion Architecture

Data Modeling

Data Warehousing

Data Lake Design

Agile Delivery

Communication

Tools

Oracle Database

Informatica PowerCenter

Git

Terraform

CI/CD

Job description

We are OGS—Omnicom's Integrated Solutions Engine that brings together a global talent community and connected capabilities in data, creative, media, commerce, and technology to build adaptive, future-ready solutions.

We are redefining the global capability center - not as an offshoring hub, but as a true inside edge. OGS is the place where the future is tested, built, and scaled for the world’s most ambitious agencies and clients.

With over 4300+ talented professionals in India, we are proud to be a diverse, inclusive, and collaborative community that thrives on innovation and excellence. Our teams work with some of the world’s most iconic brands, helping them move faster and achieve more—everywhere. We are growing rapidly and looking for passionate individuals to be part of this journey.

YOUR ROLE

We want to ensure that OGS continues to be the first choice for customers seeking innovative marketing technology, data, and analytics solutions. Our goal is to be recognized as a leader in technical excellence, delivering scalable and modern data platforms that drive business value for our clients.

This role sits within the Data & Technology arm of our business, specifically within the Tech Delivery team.

We are seeking a highly skilled and hands-on Senior Databricks Data Engineer to join a strategic data transformation programme focused on migrating a large-scale enterprise data platform from Informatica to Azure Databricks. The successful candidate will play a critical role in modernizing data ingestion, transformation, orchestration, and analytics capabilities while ensuring continuity of business-critical data processes.

This role requires strong technical expertise in Azure Databricks, PySpark, SQL, Oracle databases, and modern Data Lakehouse architectures. The ideal candidate will be capable of independently delivering complex solutions from design through implementation while collaborating with architects, business stakeholders, and cross-functional delivery teams.

YOUR RESPONSIBILITIES

• Lead the analysis, design, development, testing, and deployment of data engineering solutions on Azure Databricks.
• Participate in the migration of existing ETL/ELT workloads from Informatica to Azure Databricks.
• Analyse Informatica mappings, workflows, and dependencies and translate them into scalable Databricks solutions.
• Design and develop performant batch and streaming data pipelines using PySpark and SQL.
• Build and maintain data ingestion frameworks for structured and semi-structured data sources.
• Develop, optimize, and maintain Delta Lake-based data architectures using Medallion Architecture principles (Bronze, Silver, Gold).
• Work closely with Solution Architects and Business Analysts to understand business requirements and translate them into technical solutions.
• Implement data quality, reconciliation, validation, and monitoring processes throughout the migration lifecycle.
• Create technical specifications, design documents, and operational support documentation.
• Participate in code reviews and enforce engineering best practices and development standards.
• Support deployment activities through CI/CD pipelines and Infrastructure-as-Code frameworks.
• Collaborate with stakeholders, project managers, and delivery teams to ensure successful project outcomes.
• Mentor junior engineers and provide technical leadership across the project team.
• Support production releases and troubleshoot complex data issues when required.

SKILLS AND EXPERIENCE

• 4-6 years of hands-on experience with Azure Databricks in enterprise environments.
• Proven experience delivering large-scale data migration and modernization projects.
• Strong experience migrating ETL processes from Informatica or similar ETL platforms to cloud-native architectures.
• Strong understanding of Lakehouse architecture and modern data engineering principles.
• Extensive experience in Data Modelling, Data Warehousing, and Data Lake design.
• Experience working with large-scale enterprise data platforms.
• Strong analytical and problem-solving abilities.
• Experience working with Agile delivery methodologies.
• Ability to communicate complex technical concepts to both technical and non-technical stakeholders.
• Experience working in client-facing environments and collaborating with business stakeholders.Strong understanding of software development lifecycle and engineering best practices.

ESSENTIAL SKILLS

Azure Databricks, Databricks Genie AI, PySpark, Spark SQL, Delta Lake, Unity Catalog, Databricks Workflows, Databricks Asset Bundles, Lakehouse Architecture, Medallion Architecture, Advanced Data Engineering patterns and best practices

Database Technologies

Oracle Database (Must Have), Oracle SQL, Oracle Stored Procedures, Oracle Data Modelling, Oracle Data Warehouse Concepts and Ms SQL server

Migration & Integration

• Informatica PowerCenter or Informatica Intelligent Cloud Services (IICS)
• ETL to ELT migration methodologies
• Data Validation and Reconciliation
• Data Quality Frameworks
• Source-to-Target Mapping Analysis
• Data Integration DesignGit
• CI/CD
• Terraform
• Infrastructure as Code
• Unit Testing and Automated Testing
• Performance Optimization

DESIRABLE / BENEFICIAL SKILLS / CERTIFICATIONS

• Experience with Databricks LakeBridge, GPT, Claude AI.
• Databricks certification(s).

Shift - 2 to 11 pm

Hybrid model

Immediate joiners are preferable or less than 30 days notice period