GCP Data Engineer

Dentsu Global Services; Bengaluru; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

We are seeking a Data Engineer to join our Bengaluru-based team to design and maintain scalable data pipelines on GCP, focusing on data quality and analytics support.

Responsibilities include building ETL/ELT processes, real-time streaming with Pub/Sub, Airflow orchestration, and CI/CD with Cloud Build. Strong Python OOP skills and collaboration with data science, product, engineering, and compliance teams are required.

Qualifications

• Hands-on experience with GCP services including BigQuery, Cloud Functions, Cloud Run, Cloud Composer, GCS, and Pub/Sub.
• Experience with Airflow.
• Design real-time event-driven architectures using Pub/Sub and Eventarc.
• REST API ingestion experience.
• Strong data governance, quality, privacy, and security practices.
• CI/CD automation using Cloud Build.
• Collaborates with data science, product, engineering, and compliance teams.
• Strong Python with solid OOP principles and design patterns.
• Problem-solving and system design skills.

Responsibilities

• Design, build, and optimize scalable ETL/ELT data pipelines across GCP.
• Develop modular Python applications for data ingestion and orchestration.
• Architect and maintain data solutions using GCP services.
• Integrate external APIs to ingest datasets into BigQuery and other storage layers.
• Implement event-driven and real-time data processing architectures.
• Build CI/CD workflows using Cloud Build.
• Ensure performance, scalability, and cost efficiency of pipelines and data infrastructure.
• Establish data quality, governance, lineage, and security standards.
• Collaborate with cross-functional teams to meet business needs.
• Maintain technical documentation and best practices.

Skills

GCP BigQuery

Python

Airflow

Pub/Sub

Data pipelines

REST APIs

CI/CD

Data governance

Job description

Required Skills : SQL, Python, GCP, Big query, Pub Sub etc

Notice Period : Immediate to 30 Days

We are seeking a highly skilled and motivated Data Engineer to join our team and help build robust, scalable, and efficient data infrastructure. You will play a key role in designing and maintaining data pipelines, ensuring data quality, and supporting analytics and AI initiatives.

Key Responsibilities

Design, build, and optimize scalable ETL/ELT data pipelines across GCP.

Develop modular, object‑oriented Python applications for data ingestion, orchestration, and transformation.

Architect, deploy, and maintain data solutions leveraging GCP services: BigQuery, GCS, Cloud Functions, Cloud Run, Cloud Composer, Pub/Sub, Eventarc, and DataForm.

Integrate with external APIs to ingest datasets into BigQuery and other storage layers.

Implement event-driven and real-time data processing architectures.

Build and automate CI/CD workflows for deploying pipelines and services using Cloud Build.

Ensure optimal performance, scalability, and cost efficiency of pipelines and data infrastructure.

Establish and enforce data quality, governance, lineage, and security standards.

Work closely with cross-functional teams to convert business needs into scalable data solutions.

Maintain detailed technical documentation and contribute to engineering best practices.

Must have skills

Hands-on expertise with GCP services such as BigQuery, Cloud Functions, Cloud Run, Cloud Composer, GCS, and Pub/Sub.

Hands-on Experience in Airflow

Ability to design and implement real-time event-driven architectures using Pub/Sub and Eventarc.

Experience integrating and consuming REST APIs for data ingestion.

Strong understanding of data governance, quality, privacy, and security practices.

Experience automating builds and deployments using CI/CD pipelines (Cloud Build preferred).

Ability to collaborate effectively with data science, product, engineering, and compliance teams.

Strong proficiency in Python with solid OOP principles, including encapsulation, inheritance, polymorphism, abstraction, SOLID, and design patterns.

Strong problem-solving, debugging, and system design skills.

Good to have skills

Familiarity with DataForm for SQL-based data transformations.

Exposure to Dataflow (Apache Beam) for large-scale batch/stream processing.

Knowledge of cost‑optimization techniques for cloud workloads.

Experience with API testing tools like Postman for documentation and QA.

Understanding of software architecture patterns (e.g., clean architecture, microservices).

Experience in building reusable internal libraries or frameworks.

Experience with Vertex AI for ML workflow integration.