Lead Data Engineer

BuildxPartners; Chennai District; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

BuildxPartners in Chennai is seeking a Lead Data Engineer to design and optimize scalable cloud-based data pipelines using GCP, Dataflow, and BigQuery. You will implement robust back-end processing with Java and ensure data quality and governance across datasets.

The role involves building batch and streaming pipelines, integrating multiple data sources, and collaborating with Architects, Analysts, DevOps, and business stakeholders to deliver enterprise-grade analytics solutions.

Qualifications

• Bachelor’s degree in IT / Computer Science / Engineering / related field

Responsibilities

• Design and develop scalable ETL/ELT pipelines using GCP services.
• Build and maintain real-time and batch data processing pipelines using Dataflow and Apache Beam.
• Develop robust backend processing solutions using Java.
• Work extensively with BigQuery for data warehousing, analytics, and performance optimization.
• Integrate data from multiple sources including APIs, databases, and streaming platforms.
• Optimize pipeline performance, cost, scalability, and reliability on GCP.
• Collaborate with cross-functional teams including Architects, Analysts, DevOps, and Business stakeholders.
• Ensure data quality, governance, monitoring, and security best practices.
• Troubleshoot production issues and provide long-term scalable solutions.

Skills

GCP

Dataflow/Beam

Java

BigQuery

Batch & Streaming

SQL

CI/CD

Git

Agile

Debugging

Education

Bachelor's degree in IT/CS/Engineering

Tools

Job description

• Other Locations Bangalore; Coimbatore; Pune

About Us

BuildxPartners is a global talent solutions firm delivering end-to-end recruitment and workforce solutions across industries and geographies.

→ BuildxAlpha – Executive & Leadership Search Focused on C-suite, board, and global executive hiring.

→ BuildxSigma – Comprehensive Talent Across Levels Covering junior, mid-level, and senior professionals.

→ BuildxGCC – Global Capability Center Solutions Specializing in Build, Operate, Transfer (BOT) model for Global Capability Centers, GCC supports companies in setting up, scaling, and transferring GCCs.

Job Description

About the Company:

Our client is a fast-growing technology-driven organization focused on building scalable, cloud-based data platforms and modern analytics solutions. The company specializes in leveraging cutting-edge cloud technologies and data engineering practices to enable data-driven decision-making. With a strong focus on innovation, data quality, and governance, the organization fosters a collaborative and high-performance engineering culture.

Role Overview:

We are looking for a highly skilled Lead Data Engineer with strong expertise in Google Cloud Platform (GCP), Apache Beam/Dataflow, Java, and BigQuery. The ideal candidate will be responsible for designing, developing, and optimizing scalable cloud-based data processing pipelines for enterprise-grade applications.

Key Responsibilities:

Design and develop scalable ETL/ELT pipelines using GCP services.

Build and maintain real-time and batch data processing pipelines using Dataflow and Apache Beam.

Develop robust backend processing solutions using Java.

Work extensively with BigQuery for data warehousing, analytics, and performance optimization.

Integrate data from multiple sources including APIs, databases, and streaming platforms.

Optimize pipeline performance, cost, scalability, and reliability on GCP.

Collaborate with cross-functional teams including Architects, Analysts, DevOps, and Business stakeholders.

Ensure data quality, governance, monitoring, and security best practices.

Troubleshoot production issues and provide long-term scalable solutions.

Required Skills:

Strong hands-on experience with GCP services.

Expertise in Google Dataflow / Apache Beam.

Strong programming skills in Java.

Hands-on experience with BigQuery.

Experience in building batch and streaming data pipelines.

Good knowledge of SQL and data modelling concepts.

Experience with CI/CD, Git, and Agile methodologies.

Strong problem-solving and debugging skills.

Eligibility / Qualifications:

Education: Bachelor’s degree in IT / Computer Science / Engineering / related field