Senior Data Engineer - Hands-On Lakehouse (Databricks)

Redcat Digital; Greater London; Hybrid; GBP 100,000 - 140,000; Full time

Job summary

Redcat Digital in London is seeking an experienced Senior Data Engineer to join a large-scale data environment. This hands-on role requires coding 30–50% of the time and technical ownership across delivery, with collaboration across engineers and stakeholders.

You will work with Python, Databricks and Spark to design, develop and optimise data pipelines and lakehouse architectures. The role combines hands-on development with technical leadership in an Agile setting.

Qualifications

• 10+ years' experience across data engineering and/or software engineering.
• Proven experience as a Senior Data Engineer or Technical Lead.
• Hands-on Python development experience is essential.
• Extensive experience with Databricks (notebooks, workflows, Delta Lake).
• Strong Spark / PySpark experience.
• Experience building and optimizing large-scale ETL/ELT pipelines.

Responsibilities

• Contribute to design, development and optimisation of data pipelines using Python, PySpark and Databricks.
• Build and enhance ETL/ELT processes and reusable data engineering frameworks.
• Troubleshoot complex data and performance issues; provide hands-on production support.
• Code reviews, debugging, testing and performance optimisation.
• Own delivery of data engineering initiatives; coordinate with stakeholders.

Skills

Python

Databricks

Spark / PySpark

SQL

Cloud platforms

Data modeling

CI/CD / DevOps

Tools

Databricks

Docker

Kubernetes

Kafka

Job description

Redcat Digital in London is seeking an experienced Senior Data Engineer to join a large-scale data environment. This hands-on role requires coding 30–50% of the time and technical ownership across delivery, with collaboration across engineers and stakeholders.

You will work with Python, Databricks and Spark to design, develop and optimise data pipelines and lakehouse architectures. The role combines hands-on development with technical leadership in an Agile setting.