Technical Lead – Multi-tenant Data Integration Platform - GCP

Vallum Associates; South Yorkshire; Hybrid; GBP 80,000 - 120,000; Full time

Job summary

Vallum Associates seeks a Technical Lead for a multi-tenant data integration platform on GCP. The role is based in Sheffield with a 2-3 day onsite pattern and a mix of remote work. You will guide engineering teams, architect complex data pipelines, and ensure cloud-native reliability and security.

The role requires strong hands-on GCP experience, Airflow, Python, and IaC (Terraform), with a focus on scalable, production-grade ETL/ELT pipelines and observability.

Qualifications

• Proven experience as a Technical Lead / Lead Engineer delivering data engineering platforms in cloud environments.
• Strong hands-on experience building production-grade ETL/ELT pipelines and orchestration with Apache Airflow.
• Solid Google Cloud Platform expertise (BigQuery, Cloud Storage, Pub/Sub, Dataflow) and container platforms (GKE, Cloud Run).
• Strong software engineering fundamentals: Python, APIs, configuration management, testing, and performance tuning.
• CI/CD pipelines and Infrastructure-as-Code (Terraform) with release controls.
• Experience designing for reliability: retries, idempotency, dead-letter handling, backpressure, resilience.
• Ability to translate complex technical topics for mixed audiences and drive alignment across teams.

Skills

Technical Leadership

Data integration

ETL/ELT pipelines

Python

CI/CD

Terraform

Airflow

GCP

Tools

Terraform

Apache Airflow

GKE

Cloud Run

BigQuery

Dataflow

Pub/Sub

IAM & Networking

Job description

The Role:

Technical Lead – Multi-tenant Data Integration Platform - GCP

Location:

Sheffield, UK

Position Type:

Contract Inside IR35

Remote work option Available:

2-3 Days Onsite in a week

Job Description:

Required experience & skills

• Proven experience as a Technical Lead / Lead Engineer delivering data engineering or integration platforms in cloud environments.
• Strong hands-on experience building production-grade ETL/ELT pipelines and orchestration with Apache Airflow.
• Solid experience on Google Cloud Platform, such as:
  • Data services (e.g., BigQuery, Cloud Storage, Pub/Sub, Dataflow) and/or container platforms (e.g., GKE, Cloud Run)
  • IAM, networking, secrets management, and environment separation
• Strong software engineering foundations: Python, APIs, configuration management, testing strategy, and performance tuning.
• CI/CD expertise (pipeline tooling, release controls, automated quality gates) and Infrastructure-as-Code experience (e.g., Terraform).
• Experience designing for reliability retries, idempotency, dead-letter patterns, backpressure handling, and operational resilience.
• Ability to translate complex technical topics for mixed audiences and drive alignment across teams.

Desirable:

• Experience implementing or integrating data lineage/metadata management tooling (e.g., Data Catalog-style metadata, OpenLineage, DataHub, etc.).
• Experience with multi-tenant platforms (tenant isolation models, metadata-driven frameworks).
• Knowledge of data governance concepts (data quality, auditability, retention, access controls).
• Experience integrating with external vendor systems and managing vendor technical dependencies