Senior Data Engineering Lead - Cloud Pipelines & Governance

FalconSmartIT; Wimbledon; On-site; GBP 80,000 - 110,000; Full time

Job summary

A leading technology consultancy in the UK seeks a Big Data Lead to oversee the development of automated data pipelines using AWS and Databricks. This role demands senior expertise in Data Engineering, along with skills in building efficient ETL processes and managing a team of data engineers. The ideal candidate will have strong experience in cloud technologies and data governance. This is a contract position located in Wimbledon.

Qualifications

• Senior experience in Data Engineering and building automated data pipelines.
• Expertise in managing and scaling Databricks environments.
• Extensive experience with AWS services and DevOps practices.

Responsibilities

• Lead Data Engineering teams in designing and maintaining data infrastructures.
• Architect and manage data platforms to support real-time and batch data processing.
• Drive automation of data pipelines and improve efficiency.

Skills

Databricks

AWS Cloud services

IBM DB2

Python

SQL

ETL frameworks

DevOps practices

Apache Spark

Test automation

Team Management

Tools

IBM Datastage ETL

AWS Glue

Terraform

CloudFormation

Job description

A leading technology consultancy in the UK seeks a Big Data Lead to oversee the development of automated data pipelines using AWS and Databricks. This role demands senior expertise in Data Engineering, along with skills in building efficient ETL processes and managing a team of data engineers. The ideal candidate will have strong experience in cloud technologies and data governance. This is a contract position located in Wimbledon.