Data Engineer – Tech Lead

Jobtailor; Greater London; On-site; GBP 90,000 - 140,000; Full time

Job summary

Jobtailor in the United Kingdom seeks a senior data engineering leader to own and optimize our data platform. You will shape the direction of a growing team, coach members on best practices, and ensure reliable, scalable data pipelines that power business insights.

Collaborate with senior leadership, build new data sources, deploy ML infrastructure, and drive governance and data quality across streaming and batch workloads.

Qualifications

• Experience leading data engineering teams and mentoring engineers.
• Ability to own problems end‑to‑end in ambiguous contexts.
• Commitment to high‑quality code and lean processes.
• Experience with distributed data processing and data quality monitoring.
• Familiarity with data warehousing and API integration.

Responsibilities

• Own and optimise platform architecture for reliability and scalability.
• Shape team direction and coach members on best practices.
• Deliver data pipelines and new data sources, including API integrations.
• Interact with Senior Leadership to present data projects.
• Develop and deploy ML infrastructure for AI initiatives.
• Collaborate with dev teams to integrate data platform with back‑end databases.
• Scope and deliver data engineering projects with stakeholders.
• Build data sources and pipelines with third‑party integrations.

Skills

Data pipeline development

Team management and coaching

Distributed data processing

Python programming

AWS infrastructure

Tools

Databricks

Airflow

Dbt

Kubernetes

Spark

Job description

• Own and optimise our platform architecture (improve reliability, scalability, maintainability, etc)
• Shape the direction of our growing team and coach team members on best practices.
• Shape engineering outputs and raising the technical bar of the organization without losing touch with hands‑on engineering
• Interact with Senior Leadership to present your data projects back to them
• Build new data sources and data pipelines that deliver key data and insights to the business (this will include integration with third party systems and ingestion through APIs)
• Scope and deliver new data engineering projects in collaboration with business stakeholders
• Develop and deploy ML infrastructure to help build out our ever-growing AI requirements and use cases
• Collaborate closely with our dev teams to build seamless integrations between our back‑end databases and our data platform

Requirements

• First and foremost, a passion for decarbonisation
• Ability to work with ambiguity and own problems end‑to‑end
• Experience managing/mentoring/coaching high performance data teams
• A passion for writing high quality code and building lean processes
• Experience with distributed data processing
• Experience with monitoring, testing, and data quality
• Experience building robust pipelines from diverse sources e.g. parquet, SQL & no‑SQL databases, API endpoints
• It would be helpful to have experience/expertise/knowledge in the following:
• Python
• AWS
• Kubernetes
• Spark & distributed computing
• dbt
• Terraform
• Data Warehousing (we use Databricks but experience with other platforms is fine)
• MLOps implementation (desirable but not required)
• Airflow (or other orchestration tooling)
• Governance

Core Competencies

Demonstrates expertise in data engineering, including building and optimizing data pipelines, managing high-performance teams, and developing machine learning infrastructure. Proficient in integrating diverse data sources and ensuring data quality while collaborating with stakeholders to deliver impactful insights.

Highest-signal resume keywords

• Data Pipeline Development
• Team Management and Coaching
• Distributed Data Processing
• Python Programming
• AWS Infrastructure

ATS Optimization Keywords

Hard Skills

• Data Engineering
• Data Quality Monitoring
• Data Warehousing
• MLOps Implementation
• Distributed Computing
• API Integration
• SQL Databases
• No-SQL Databases
• Parquet
• Terraform

Soft Skills

• Problem Solving
• Collaboration
• Adaptability
• Communication
• Coaching

Industry Keywords

• Decarbonisation
• Governance
• Data Architecture
• Machine Learning
• Data Integration

Tools & Technologies

• AWS
• Kubernetes
• Spark
• Dbt
• Databricks
• Airflow