Lead Data Engineer - GCP

Relanto; Fremont (CA); On-site; USD 120,000 - 150,000; Full time

Job summary

Relanto is seeking an experienced Data Engineer to design and implement scalable, secure data platforms on GCP. This role demands strong expertise in the GCP data ecosystem, specifically in batch and streaming data pipelines using PySpark and Python.

The ideal candidate will lead technical discussions, collaborate with stakeholders, and mentor junior engineers, ensuring compliance with governance standards.

A solid understanding of data architectures and hands-on experience with tools such as Apache Airflow and BigQuery is essential.

Qualifications

• 6–9 years of experience in Data Engineering.
• Strong understanding of distributed systems and scalable data processing.
• Expertise in data modeling, partitioning, clustering, and query optimization.

Responsibilities

• Design and architect scalable, secure, and high-performance data platforms on GCP.
• Lead the development of large-scale batch and streaming data pipelines using PySpark and Python.
• Collaborate with various stakeholders to translate business requirements into technical solutions.

Skills

GCP data ecosystem

Python

PySpark

SQL

Apache Airflow / Cloud Composer

dbt

BigQuery

Tools

Terraform

GitOps

Cloud Storage

Dataflow

Dataproc

IAM

Job description

Responsibilities

• Design and architect scalable, secure, and high-performance data platforms on GCP
• Lead the development of large-scale batch and streaming data pipelines using PySpark and Python
• Build and optimize orchestration frameworks using Apache Airflow/Cloud Composer
• Design enterprise-grade data models and transformation frameworks using dbt
• Implement scalable data warehouse and lakehouse solutions using BigQuery
• Define architecture standards, engineering best practices, and governance frameworks
• Drive performance optimization, scalability, and cloud cost optimization initiatives
• Collaborate with Architects, Product Owners, Analysts, and business stakeholders to translate business requirements into technical solutions
• Lead technical design discussions, code reviews, and architecture reviews
• Implement observability, monitoring, lineage, and data quality frameworks
• Mentor junior engineers and provide technical leadership to the team
• Ensure compliance with security, governance, and regulatory standards
• Support CI/CD and Infrastructure as Code implementations

Required Skills

• 6–9 years of experience in Data Engineering
• Strong expertise in GCP data ecosystem and cloud-native architectures
• Deep hands-on experience with:
  • Python
  • PySpark
  • SQL
  • Apache Airflow / Cloud Composer
  • dbt
  • BigQuery
• Strong understanding of data architecture patterns including:
  • Lakehouse Architecture
  • Medallion Architecture
  • Data Mesh concepts
  • Batch and Streaming architectures
• Experience with GCP services such as:
  • BigQuery
  • Dataproc
  • Cloud Storage
  • Pub/Sub
  • Dataflow
  • Composer
  • IAM
• Expertise in data modeling, partitioning, clustering, and query optimization
• Strong understanding of distributed systems and scalable data processing
• Experience with CI/CD, GitOps, Terraform, and containerization technologies
• Excellent stakeholder management and technical leadership skills