GCP Data Engineering Lead - BigQuery & Dataflow

Capgemini; United States; Remote; USD 72,000 - 110,000; Full time

Job summary

Tundra Technical Solutions on behalf of Capgemini in the United States seeks a Lead Data Engineer to guide a team of 10 engineers onshore and offshore, shaping data pipelines and platform standards.

You'll own end-to-end data engineering, using GCP with BigQuery, Dataflow, Dataproc, Pub/Sub, Airflow, and Terraform, driving performance, cost optimization, and collaboration with architects, scientists, and analysts on advanced analytics and ML use cases.

Qualifications

• Leadership experience directing data engineering teams onshore/offshore.
• Hands-on with SQL, Python, Java/Scala; knowledge of distributed systems (Spark/Beam).
• GCP expertise with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer, Terraform.
• DevOps familiarity with CI/CD, Git and infra-as-code.

Responsibilities

• Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow, Dataproc, Pub/Sub, and Cloud Functions.
• Data Infrastructure: Architect and manage BigQuery, Cloud Storage, and Data Fusion for ingestion, transformation, and storage.
• Team Leadership: Lead a team of 10 engineers, review code, establish data eng standards, CI/CD, and monitoring.
• Performance & Cost: Optimize pipeline performance, query efficiency, and GC/GCP cost management.
• Collaboration: Work with data architects, scientists, and analysts on self-service data platforms for analytics/ML.

Skills

Python

Shell scripting

Makefiles

CI/CD

SQL

Java/Scala

Distributed systems

Spark

Beam

BigQuery

Dataflow

Dataproc

Pub/Sub

Composer

Terraform

Git

Data governance

Tools

Terraform

dbt

Job description

Tundra Technical Solutions on behalf of Capgemini in the United States seeks a Lead Data Engineer to guide a team of 10 engineers onshore and offshore, shaping data pipelines and platform standards.

You'll own end-to-end data engineering, using GCP with BigQuery, Dataflow, Dataproc, Pub/Sub, Airflow, and Terraform, driving performance, cost optimization, and collaboration with architects, scientists, and analysts on advanced analytics and ML use cases.