Lead Data Engineer

Wipro; Chennai; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Wipro is seeking a Lead Data Engineer in Chennai to drive modernization of the Sales Validation platform and design future-state architecture using Databricks and Azure cloud technologies.

You will lead end-to-end delivery, mentor engineers, and collaborate with data scientists, architects, and business stakeholders to implement scalable data solutions and improve performance across pipelines.

Qualifications

• Lead architecture, implementation, and delivery of Databricks solutions for data processing and analytics.
• Oversee migration from Synapse Analytics to Databricks with minimal downtime.
• Develop engineering standards, reusable frameworks, and documentation.
• Mentor data engineers through design reviews and knowledge sharing.
• Own end-to-end delivery of multiple workstreams and manage risks.
• Drive performance tuning and scalability of data pipelines.
• Collaborate with data scientists, architects, and stakeholders to deliver aligned solutions.

Responsibilities

• Gather detailed structured requirements for data engineering modules and document them.
• Lead solution design workshops to align architecture with business objectives and compliance.

Skills

Client Centricity

Passion for Results

Execution Excellence

Collaborative Working

Learning Agility

Problem Solving & Decision Making

Effective Communication

Tools

Databricks

Spark

Delta Lake

Unity Catalog

Azure Data Factory

ADLS Gen2

Azure SQL

Apache Airflow

Snowflake

GitLab CI/CD

Job description

Job Description:

Lead data engineer

Lead Data Engineer

Role Summary

We are seeking a highly skilled and experienced Lead Data Engineer to drive the modernization and design the future-state architecture of our Sales Validation platform. As a technical leader, you will be responsible for guiding the transition from legacy systems to advanced data engineering solutions, primarily focused on Databricks and Azure cloud technologies. This role requires a passion for innovation, performance optimization, and building scalable, reusable engineering frameworks.

Key Responsibilities

• Lead the architecture, implementation, and delivery of Databricks solutions to support data processing and analytics initiatives.


• Oversee the migration efforts from Synapse Analytics to Databricks, ensuring minimal downtime and aligned best practices.


• Develop and enforce engineering standards, reusable frameworks, and documentation to promote consistency and efficiency across teams.


• Mentor data engineers through design reviews, knowledge sharing sessions, and technical guidance to foster professional growth and code quality.


• Own end-to-end delivery of multiple complex workstreams, coordinate across teams, and manage technical risks.


• Drive continuous performance tuning and scalability improvements for data pipelines and platforms.


• Collaborate with cross-functional teams including data scientists, architects, and business stakeholders to deliver aligned solutions.

Core Competencies

• Client Centricity: Demonstrate a strong commitment to understanding and fulfilling client needs, ensuring solutions deliver measurable value.


• Passion for Results: Exhibit a persistent focus on achieving outcomes through dedication and proactive problem-solving.


• Execution Excellence: Deliver consistent quality and meet deadlines through disciplined planning and thorough validation.


• Collaborative Working: Foster teamwork and open communication across diverse stakeholders and disciplines.


• Learning Agility: Embrace change and continuously seek to expand knowledge and skills in evolving technologies.


• Problem Solving & Decision Making: Analyze complex issues, identify root causes, and implement effective, timely solutions.


• Effective Communication: Articulate ideas clearly and persuasively in both technical and non-technical contexts to diverse audiences.

Profile description:

Technical Skills Required

• Advanced experience with Databricks and Spark frameworks using PySpark and Spark SQL.


• Deep understanding of Delta Lake architecture and Unity Catalog for secure data governance.


• Practical knowledge of Databricks workflows and asset bundles to orchestrate data pipelines effectively.


• Expertise with modern Lakehouse architecture principles to integrate data lakes and warehouses.

Microsoft Azure Platform

• Hands-on experience architecting and managing Azure Data Factory (ADF) pipelines and ecosystem components.


• Proficiency with Azure Data Lake Storage Gen2 (ADLS Gen2) for secure and scalable storage solutions.


• Strong familiarity with Azure SQL databases and implementation of Azure security best practices.

Migration and Modernization

• Experience migrating Synapse Pipelines, SQL Pools, Notebooks, and Data Flows into scalable Databricks platforms.


• Successful track record in modernizing legacy ETL workloads for improved reliability and performance.

Software Engineering & Development

• Advanced SQL skills, data modeling expertise, and performance tuning abilities.


• Experience with GitLab CI/CD pipelines for automation of testing, deployment, and code quality checks.


• Strong adherence to code review best practices and collaborative development methodologies.

Preferred Skills and Experience

• Familiarity with Apache Airflow orchestration for task scheduling and workflow management.


• Exposure to Snowflake data platform technologies and data governance strategies.


• Experience deploying enterprise-scale Azure platforms with considerations for security, scalability, and operational excellence.

Areas of Responsibility

In this role, you will participate actively in gathering detailed and structured requirements for various data engineering modules. You will engage with multiple business units to understand dependencies, process linkages, and end-user needs ensuring comprehensive and clear documentation.

You will also be involved in facilitating solution design workshops under project leadership guidance, ensuring the technical architecture aligns with both business objectives and compliance standards. This includes translating functional requirements into robust, scalable data solutions.