Senior Data Engineer - Spark & PySpark, Databricks (Remote)

Xcede; Grimsby; Remote; GBP 90,000 - 120,000; Full time

Job summary

A leading data technology firm is seeking a Senior Data Engineer to design and build large-scale distributed data pipelines. This fully remote role requires strong Python and Spark skills, along with experience in cloud infrastructure. You will work independently while mentoring other engineers and optimizing data workflows. Ideal candidates will have real-world experience with Databricks and a proven track record in handling large-scale data processing in regulated industries.

Qualifications

• Proven experience as a Senior Data Engineer.
• Strong Python software engineering foundation.
• Hands-on Spark experience in production (PySpark essential).
• Real-world experience using Databricks for data pipelines.

Responsibilities

• Design, build, and evolve large-scale distributed data pipelines using Spark/PySpark.
• Develop production-grade Python data workflows.
• Optimize cloud-based data infrastructure (AWS preferred).
• Collaborate with engineers on client-facing projects.
• Support and mentor other engineers.

Skills

Python

Spark

Databricks

Cloud Infrastructure (AWS/Azure)

Large-scale data processing

Job description

A leading data technology firm is seeking a Senior Data Engineer to design and build large-scale distributed data pipelines. This fully remote role requires strong Python and Spark skills, along with experience in cloud infrastructure. You will work independently while mentoring other engineers and optimizing data workflows. Ideal candidates will have real-world experience with Databricks and a proven track record in handling large-scale data processing in regulated industries.