GCP Senior Data Engineer

Horizontal Talent; Bengaluru; Presencial; INR 3.500.000 - 6.000.000; Jornada completa

Descripción de la vacante

Horizontal Talent is seeking a Senior GCP Data Engineer for a key client. You will design, build, and maintain scalable data pipelines on Google Cloud Platform, and develop end-to-end ETL/ELT workflows for ingestion, transformation, and storage.

You will work with BigQuery, Cloud Storage, Cloud Run, Data Fusion, Dataproc, Composer, and Pub/Sub, using SQL, Python, and PySpark to deliver production-grade data solutions.

Formación

• 5+ years of total experience in data engineering, data platforms, or data-centric software engineering.
• 5+ years of hands-on experience designing and building data pipelines, ETL/ELT workflows, and cloud-native data solutions.
• 3+ years of direct experience with Google Cloud Platform (GCP) using services such as BigQuery, Cloud Storage, Cloud Run, Datafusion, Dataproc, Composer, and Pub/Sub.
• Advanced proficiency in SQL, Python, and PySpark for data processing and transformation.
• Proven experience developing and supporting production-grade, end-to-end cloud data pipelines.
• Expertise in ELT/ETL design, performance optimization, and data transformation frameworks.
• Strong background in monitoring, logging, alerting, and data quality frameworks across distributed systems.
• Experience with DevOps tools and techniques, including CI/CD, GitHub, and related automation practices.
• Demonstrated experience working in global, cross-functional, and agile development environments.
• Excellent communication, problem-solving, and analytical skills with the ability to collaborate across technical and non-technical teams.
• GCP certification preferred (e.g., Google Cloud Professional Data Engineer).

Responsabilidades

• Design, build, and maintain scalable data pipelines and cloud-native data platforms on GCP.
• Develop end-to-end ETL/ELT workflows for data ingestion, transformation, and storage.
• Work with BigQuery, Cloud Storage, Cloud Run, Data Fusion, Dataproc, Composer, and Pub/Sub.
• Build production-grade data solutions using SQL, Python, and PySpark.
• Optimize pipelines for performance, scalability, cost, and reliability.
• Implement monitoring, logging, alerting, and data-quality frameworks.
• Collaborate with Data Scientists, Analysts, DevOps, Platform Engineering, and business stakeholders.
• Follow CI/CD, GitHub, Agile, coding standards, and engineering best practices.

Conocimientos

Problem-solving

Analytical skills

Communication

Team collaboration

Educación

Bachelor's / Master’s degree in Computer Science or related field

Herramientas

GCP

BigQuery

Cloud Storage

Cloud Run

Data Fusion

Dataproc

Composer

Pub/Sub

SQL

Python

PySpark

CI/CD

GitHub

Descripción del empleo

Horizontal Talent is looking for an experienced Senior GCP Data Engineer to join one of our esteemed clients.

About Horizontal

Established since 2003 in the US, Horizontal solves complex challenges across two distinct businesses: Horizontal Digital and Horizontal Talent. We are consistently recognized for being a top workplace and one of the fastest growing private companies. Horizontal Talent specializes in staffing for IT, Digital & Creative and Business & Strategy markets. We have global offices in US, UAE, India, and Malaysia.

Notice Period: Preference will be given to immediate joiners.

What You’ll Do

• Design, build, and maintain scalable data pipelines and cloud-native data platforms on Google Cloud Platform (GCP).
• Develop end-to-end ETL/ELT workflows for data ingestion, transformation, and storage.
• Work with BigQuery, Cloud Storage, Cloud Run, Data Fusion, Dataproc, Composer, and Pub/Sub.
• Build production-grade data solutions using SQL, Python, and PySpark.
• Optimize pipelines for performance, scalability, cost, and reliability.
• Implement monitoring, logging, alerting, and data-quality frameworks.
• Collaborate with Data Scientists, Analysts, DevOps, Platform Engineering, and business stakeholders.
• Follow CI/CD, GitHub, Agile, coding standards, and engineering best practices.

Qualifications

Education Requirements

Bachelor’s /Master’s degree in Computer Science, Information Technology, Data Engineering, Software Engineering, or a related technical field.

Experience Requirements

5+ years of total experience in data engineering, data platforms, or data‑centric software engineering.

5+ years of hands‑on experience designing and building data pipelines, ETL/ELT workflows, and cloud‑native data solutions.

3+ years of direct experience with Google Cloud Platform (GCP) using services such as BigQuery, Cloud Storage, Cloud Run, Datafusion, Dataproc, Composer, and Pub/Sub.

Advanced proficiency in SQL, Python, and PySpark for data processing and transformation.

Proven experience developing and supporting production‑grade, end‑to‑end cloud data pipelines.

Expertise in ELT/ETL design, performance optimization, and data transformation frameworks.

Strong background in monitoring, logging, alerting, and data quality frameworks across distributed systems.

Experience with DevOps tools and techniques, including CI/CD, GitHub, and related automation practices.

Demonstrated experience working in global, cross‑functional, and agile development environments.

Excellent communication, problem‑solving, and analytical skills with the ability to collaborate across technical and non‑technical teams.

GCP certification preferred (e.g., Google Cloud Professional Data Engineer).