Senior Data Engineer - Hybrid (Python / Spark / Databricks)

GCS Recruitment; Greater London; Hybrid; GBP 136,000 - 178,000; Full time

Job summary

GCS Recruitment seeks a Senior Data Engineer (Python / Spark / Databricks) for a hybrid London-based contract. The role focuses on building scalable data pipelines, Databricks lakehouse architectures, and robust data platforms in a modern cloud environment.

Strong expertise in Python, Databricks, Spark/PySpark and SQL is required, with hands-on delivery across ETL/ELT, data quality and CI/CD practices. Banking domain experience is a plus; three days in the Liverpool Street office each week.

Qualifications

• Strong commercial experience as a Data Engineer or Senior Data Engineer.
• Expert-level Python development skills.
• Extensive hands-on experience with Databricks in enterprise environments.
• Strong Apache Spark and PySpark expertise.
• Advanced SQL development and optimisation skills.
• Experience designing and building large-scale ETL and data integration solutions.
• Experience with modern cloud platforms (Azure, AWS or GCP).
• Experience working in Agile environments.

Responsibilities

• Design, develop and maintain scalable data pipelines using Python, Databricks and PySpark.
• Build and optimise ETL/ELT solutions processing large-scale datasets.
• Develop high-quality, reusable code and engineering frameworks.
• Deliver robust, performant and maintainable data solutions.
• Troubleshoot and resolve complex production and performance issues.
• Support ongoing enhancement and modernisation of enterprise data platforms.
• Develop and support Databricks-based data solutions and Lakehouse architectures.
• Utilise Delta Lake and Spark technologies for scalable data processing.

Skills

Python

PySpark

Spark

SQL

Data Engineering

Agile development

Tools

Databricks

Delta Lake

Airflow

Docker

Kubernetes

CI/CD

Job description

GCS Recruitment seeks a Senior Data Engineer (Python / Spark / Databricks) for a hybrid London-based contract. The role focuses on building scalable data pipelines, Databricks lakehouse architectures, and robust data platforms in a modern cloud environment.

Strong expertise in Python, Databricks, Spark/PySpark and SQL is required, with hands-on delivery across ETL/ELT, data quality and CI/CD practices. Banking domain experience is a plus; three days in the Liverpool Street office each week.