Senior Data Engineer – GCP | BigQuery | Dataflow | Dataproc

Umanist Staffing LLC; Chennai District; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

Umanist Staffing LLC is seeking a Senior Data Engineer in Chennai, India. The role focuses on designing, building, and maintaining scalable data platforms and pipelines using GCP, BigQuery, Dataflow, and Dataproc.

You will develop ETL/ELT workflows with PySpark, Python, and build real-time and batch processing solutions. The ideal candidate has 7–11 years of data engineering experience, strong GCP expertise, and hands-on skills in Airflow, Terraform, Tekton, and PostgreSQL.

Qualifications

• 7 years of experience in Data Engineering.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Expertise in services: BigQuery, Dataflow, Dataproc, Data Fusion, Cloud SQL.
• Strong experience with Terraform and infrastructure-as-code.
• Hands-on with Tekton and CI/CD pipelines.
• Airflow workflow orchestration with PySpark-based processing.
• Python for data engineering and automation.
• Strong PostgreSQL and relational DB concepts.
• Experience with large-scale data processing systems.
• Knowledge of APIs, data integration, and streaming concepts.

Responsibilities

• Design, build, and maintain scalable data pipelines and data platforms.
• Develop and optimize data solutions using GCP services.
• Design data architectures including data warehouses and lakes.
• Build ETL/ELT pipelines for structured and unstructured data.
• Develop data transformation workflows using PySpark and Python.
• Design real-time and batch processing solutions.
• Work with BigQuery, Dataflow, Dataproc, Data Fusion, Cloud SQL.
• Develop workflows with Apache Airflow.
• Implement IaC using Terraform and CI/CD with Tekton.
• Build data models to support analytics and ML.
• Monitor, troubleshoot, and optimize data systems.
• Implement data governance, security, and compliance.
• Develop APIs and integrate with enterprise apps.
• Collaborate with data scientists, analysts, architects, stakeholders.
• Mentor junior team members.

Skills

Data engineering

GCP

PySpark

Python

Airflow

Terraform

Tekton

BigQuery

Dataflow

Dataproc

Data Fusion

Cloud SQL

PostgreSQL

Education

Bachelor's degree in Computer Science/IT/Data Engineering

Tools

BigQuery

Dataflow

Dataproc

Data Fusion

Cloud SQL

Terraform

Tekton

Airflow

PostgreSQL

Job description

Additional Important Note for Applicants

· Currently, only immediate joiners (who have already completed their notice period) or candidates serving a notice period of up to 30 days will be considered for this opportunity.

· Candidates with longer notice periods may not be considered at this stage due to urgent project requirements.

Important Note for Applicants

Senior Data Engineer — GCP | BigQuery | Dataflow | Dataproc

Location: Chennai, India

Experience Required: 7–11 Years

Job Openings: 2 Positions

Work Mode: Hybrid

Shift Type: Day Shift

Employment Type: Full-time,30088

Role Overview

We are looking for an experienced Senior Data Engineer with strong expertise in Google Cloud Platform (GCP), big data technologies, data pipeline development, and cloud-based data architecture.

The ideal candidate will be responsible for designing, developing, and maintaining scalable data platforms, data pipelines, and processing solutions. The role involves working with large-scale data systems, cloud technologies, automation frameworks, and collaborating with cross-functional teams to deliver reliable and efficient data solutions.

Key Responsibilities

• Design, build, and maintain scalable data engineering solutions including data pipelines, data platforms, and data processing frameworks.
• Develop and optimize data solutions using Google Cloud Platform services.
• Design and implement data architectures including data warehouses, data lakes, and lakehouse solutions.
• Build and maintain ETL/ELT pipelines for structured and unstructured data processing.
• Develop data transformation workflows using PySpark and Python.
• Design and implement real-time and batch data processing solutions.
• Work with cloud data services including BigQuery, Dataflow, Dataproc, Data Fusion, and Cloud SQL.
• Develop and manage workflow orchestration using Apache Airflow.
• Implement infrastructure automation using Terraform and CI/CD practices using Tekton.
• Build and optimize data models to support analytics, machine learning, and business intelligence requirements.
• Monitor, troubleshoot, and optimize data systems for performance, scalability, and reliability.
• Implement data governance, security, and compliance best practices.
• Develop APIs and integrate data solutions with enterprise applications.
• Collaborate with data scientists, analysts, architects, and business stakeholders.
• Mentor junior team members and contribute to engineering best practices.

Must-Have Skills

• 7 years of experience in Data Engineering.
• Strong hands‑on experience with Google Cloud Platform (GCP).
• Expertise in the following GCP services:
• BigQuery
• Dataflow
• Dataproc
• Data Fusion
• Cloud SQL
• Strong experience with Terraform and infrastructure-as-code practices.
• Hands‑on experience with Tekton and CI/CD pipeline implementation.
• Strong knowledge of Apache Airflow workflow orchestration.
• Experience working with Airflow and PySpark‑based data processing workflows.
• Proficiency in Python for data engineering and automation.
• Strong experience with PostgreSQL and relational database concepts.
• Experience with data pipeline development, optimization, and troubleshooting.
• Experience working with large‑scale data processing systems.
• Good understanding of APIs, data integration, and streaming concepts.
• Strong analytical and problem‑solving skills.

Nice-to-Have Skills

• Experience with Kafka and Google Pub/Sub for real‑time data streaming.
• Experience with Jira or similar project management tools.
• Knowledge of data governance frameworks and best practices.
• Experience with machine learning platforms and model deployment workflows.
• Familiarity with data visualization and business intelligence tools.
• Experience with cloud data architecture and multi-cloud environments.
• Knowledge of containerization technologies and Kubernetes.
• Experience implementing monitoring and reliability practices for data platforms.
• Understanding of security and compliance practices in cloud environments.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or related field.
• Strong experience designing and implementing enterprise-scale data solutions.
• Ability to work independently and collaborate effectively with cross‑functional teams.
• Excellent communication skills with the ability to explain complex technical concepts.

Interview Process

• 2 rounds of interviews.
• Interview Mode: Video/Virtual.
• Candidates should be available for further discussions within a short turnaround time.

Candidate Preference

• Candidates available with a short notice period are preferred.
• Candidates with stable work experience and strong hands‑on GCP Data Engineering expertise are preferred.