Senior Data Engineer - Python, Spark & Databricks (Hybrid)

GCS; Greater London; Hybrid; GBP 37,638,000 - 39,852,000; Full time

Job summary

GCS is seeking a Senior Data Engineer to design, develop and optimize scalable data pipelines in a hybrid London setting. The role focuses on Python, Databricks, Spark/PySpark and SQL within a modern cloud environment.

You will work closely with engineers and stakeholders, participate in Agile delivery, and contribute to platform stability and data quality through best practices, testing and CI/CD.

Qualifications

• Extensive hands-on Databricks experience in enterprise environments.
• Strong Python development skills and SQL optimization.
• Experience with Azure, AWS or GCP and Agile teams.

Responsibilities

• Design, develop and maintain scalable data pipelines using Python, Databricks and PySpark.
• Build and optimise ETL/ELT solutions processing large-scale datasets.
• Develop high-quality, reusable code and engineering frameworks.
• Deliver robust, performant and maintainable data solutions.
• Troubleshoot and resolve complex production and performance issues.

Skills

Python

Databricks

Spark/PySpark

SQL

Cloud platforms

Agile

Data Engineering

Problem solving

Tools

Airflow

Docker

Kubernetes

Job description

GCS is seeking a Senior Data Engineer to design, develop and optimize scalable data pipelines in a hybrid London setting. The role focuses on Python, Databricks, Spark/PySpark and SQL within a modern cloud environment.

You will work closely with engineers and stakeholders, participate in Agile delivery, and contribute to platform stability and data quality through best practices, testing and CI/CD.