Data Integration Platform Lead

Vallum Associates; Birmingham; On-site; GBP 90,000 - 130,000; Full time

Job summary

Vallum Associates invites applications for a Technical Lead / Lead Engineer to drive data engineering and integration platforms in cloud environments. You will design and implement production-grade ETL/ELT pipelines and orchestrate with Apache Airflow.

The role requires deep GCP expertise, Python and strong CI/CD along with IaC experience using Terraform. You will work across teams to ensure reliability, performance, and scalable architectures in a Birmingham-based setting.

Qualifications

• Proven Technical Lead delivering data platforms in cloud environments.
• Hands-on ETL/ELT pipelines and Airflow orchestration.
• GCP experience with data services or container platforms.
• IAM, networking, secrets management and environment separation.
• Python, APIs, configuration management, testing and performance tuning.
• CI/CD pipelines and Terraform IaC.
• Design for reliability: retries, idempotency, dead-letter patterns, backpressure.
• Translate complex topics and align across teams.

Skills

Technical leadership

Data engineering

Python

APIs

Testing strategy

Performance tuning

CI/CD

Reliability design

Cloud concepts

Stakeholder alignment

Tools

Airflow

Google Cloud Platform

BigQuery

Cloud Storage

Pub/Sub

Dataflow

GKE

Cloud Run

Terraform

Job description

Required experience & skills

• Proven experience as a Technical Lead / Lead Engineer delivering data engineering or integration platforms in cloud environments.
• Strong hands-on experience building production-grade ETL/ELT pipelines and orchestration with Apache Airflow.
• Solid experience on Google Cloud Platform, such as:
• Data services (e.g., BigQuery, Cloud Storage, Pub/Sub, Dataflow) and/or container platforms (e.g., GKE, Cloud Run)
• IAM, networking, secrets management, and environment separation
• Strong software engineering foundations: Python, APIs, configuration management, testing strategy, and performance tuning.
• CI/CD expertise (pipeline tooling, release controls, automated quality gates) and Infrastructure-as-Code experience (e.g., Terraform).
• Experience designing for reliability: retries, idempotency, dead-letter patterns, backpressure handling, and operational resilience.
• Ability to translate complex technical topics for mixed audiences and drive alignment across teams.