Senior Data Engineer

wpp; India; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

WPP is seeking a highly skilled Senior Data Engineer to design, build, and optimize a scalable lakehouse platform using Google BigQuery or Databricks. You will own end-to-end data pipelines, ingest data from GA4, and ensure data quality across the Bronze-Silver-Gold layers in a multi-cloud environment.

You will apply SQL, Python, and PySpark to transform data, monitor performance, and collaborate with analytics teams to deliver analysis-ready datasets for business insights.

Qualifications

• Bachelor's degree required or equivalent.
• 8+ years in data engineering with distributed systems.
• Experience with lakehouse architectures and data pipelines.

Responsibilities

• Design, build, and deploy ETL/ELT pipelines in the lakehouse.
• Implement Bronze-Silver-Gold layers using Delta Lake/BigQuery.
• Ingest data from GA4 and other sources using transfer services.
• Develop and monitor multi-task data workflows with Airflow/Databricks.
• Optimize tables and queries for performance and cost.
• Collaborate with analytics and business teams.

Skills

Lakehouse

BigQuery

Databricks

PySpark

SQL

Python

Cloud (GCP/AWS/Azure)

Data governance

Education

Bachelor's degree in CS/Engineering or related field

Tools

BigQuery

Databricks

Spark

Delta Lake

Cloud Composer / Airflow

Pub/Sub

Job description

WPP is the trusted growth partner for the world's leading brands.

We unite cutting-edge media intelligence and data solutions, world-class creativity, next-generation production, transformative enterprise solutions and expert strategic counsel in a single company - powered by exceptional talent and our agentic marketing platform, WPP Open, to help our clients navigate change, capture opportunity and deliver transformational growth.

We work with the world's most valuable brands and have global reach across 100+ markets, with deep local expertise.

Our people are the key to our success. We're committed to fostering a culture of creativity, belonging and continuous learning, attracting and developing the brightest talent, and providing exciting career opportunities that help our people grow.

For more information, visit WPP.com.

Why we're hiring:

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team. In this critical role, you will be instrumental in designing, building, and optimizing our scalable data lakehouse platform using Google BigQuery or Databricks. You will be a key player in developing robust data pipelines that ingest data from various sources, including Google Analytics 4 (GA4), and transform it into reliable, analysis-ready datasets within the lakehouse environment. This role requires deep expertise in modern lakehouse platforms - Google BigQuery and/or Databricks - together with strong skills in SQL, Python, and Apache Spark (PySpark), along with strong hands‑on experience across Azure, AWS, and GCP cloud environments, as our data ecosystem spans multiple cloud platforms. You will be responsible for the entire data lifecycle within the lakehouse, from ingestion and transformation to governance and optimization, ensuring data quality and performance. You should be adept at analyzing performance bottlenecks in Spark jobs and BigQuery workloads, providing enhancement recommendations, and collaborating effectively with both technical and non-technical stakeholders.

What you'll be doing:

• Design, build, and deploy robust ETL/ELT pipelines within the lakehouse platform (Google BigQuery or Databricks) using SQL, Python, PySpark, and Spark SQL.
• Implement and manage the Medallion Architecture (Bronze, Silver, Gold layers) using Delta Lake or BigQuery datasets to ensure data quality and progressive data refinement.
• Leverage native ingestion tooling - such as BigQuery Data Transfer Service, Pub/Sub streaming, or Databricks Auto Loader - for efficient, scalable, and incremental ingestion of data from sources like GA4 into the Bronze layer.
• Develop, schedule, and monitor complex, multi‑task data workflows using Cloud Composer (Airflow), BigQuery scheduled queries, or Databricks Workflows.
• Optimize BigQuery tables (partitioning, clustering, materialised views) and Spark jobs / Delta Lake tables (using techniques like OPTIMIZE, Z‑ORDER, and partitioning) for high performance and cost efficiency.
• Implement data governance, security, and discovery using Dataplex / BigQuery policy tags or Unity Catalog, including managing access controls and data lineage.
• Write complex, customized SQL queries to manipulate data and support ad‑hoc analytical requests from business teams.
• Develop strategies for data ingestion from multiple sources, using various techniques including streaming, API consumption, and replication.
• Document data engineering processes, data models, and technical specifications for the lakehouse platform.
• Conform to agile development practices, including version control (Git), continuous integration/delivery (CI/CD), and test‑driven development.
• Provide production support for data pipelines, actively monitoring and resolving issues to ensure the continuous flow of critical data.
• Collaborate with analytics and business teams to understand data requirements and deliver well‑modelled, performant datasets in the gold layer.

What you'll need:

• Education: Minimum of a bachelor's degree in computer science, Engineering, Mathematics, or a related technical field preferred.
• Experience: 8+ years of relevant experience in data engineering, with a significant focus on building data pipelines on distributed systems.

Engineer's Core Skills:

• Lakehouse Platform Expertise (Google BigQuery and/or Databricks)
• BigQuery: Deep, hands‑on experience with BigQuery architecture, including partitioning, clustering, materialised views, slot/cost optimisation, and diagnosing query performance using query plans and INFORMATION_SCHEMA.
• Apache Spark / Delta Lake: Strong experience with Spark architecture, writing and optimising PySpark and Spark SQL jobs, and building reliable pipelines on Delta Lake. Proficient with ACID transactions, time travel, schema evolution, and DML operations (MERGE, UPDATE, DELETE).
• Data Ingestion: Experience with modern ingestion tools, such as BigQuery Data Transfer Service, Pub/Sub / Dataflow streaming, Databricks Auto Loader, and COPY INTO