Senior Data Engineer

SR2 REC LTD; Greater London; On-site; GBP 60,000 - 80,000; Full time

Job summary

A leading data consultancy is urgently seeking a Senior Data Engineer with deep Databricks expertise to lead a greenfield project within the public sector. This strategic role requires 2-3+ years of hands-on Databricks experience, along with strong Python skills for data transformation. The successful candidate will define technical architecture, own data pipelines, and collaborate with stakeholders to ensure compliance and performance. Active SC Clearance is a must for this highly visible role.

Qualifications

• Strong hands-on experience with Databricks, including end-to-end implementation.
• Track record of delivering greenfield data platforms.
• Strong Python skills for data transformation.

Responsibilities

• Act as the technical lead for a new Databricks implementation.
• Own and deliver streaming and batch data pipelines.
• Lead PoCs with hyperscalers for appropriate data services.

Skills

Databricks experience

Python skills

Understanding of modern data architectures

Ability to operate strategically and tactically

Job description

Senior Data Engineer - Databricks - SC Cleared

SR2 is supporting a critical greenfield transformation programme within the public sector and is urgently seeking a Senior Data Engineer/Architect with deep Databricks experience to help set the direction and strategy of a greenfield project.

This is a strategic role for someone who has not only delivered, but led end-to-end Databricks implementations, ideally across multiple programmes. You'll be defining technical architecture, leading Proof of Concepts (PoCs), and building out a modern data platform from scratch in a highly visible public sector environment.

Key Responsibilities:

• Act as the technical lead for a new Databricks implementation, working from greenfield through to full production deployment.
• Own and deliver streaming and batch data pipelines in Databricks for complex, sensitive use cases.
• Lead PoCs with hyperscalers to assess and select appropriate data services and tooling.
• Collaborate with engineering teams, stakeholders, and partners to ensure scalability, performance, and compliance.
• Document best practices, decisions, and technical architecture to support future scaling and handover.

Essential Experience:

• 2-3+ years of strong, hands-on Databricks experience, including having led implementations from setup through to production.
• Demonstrable track record of delivering greenfield or ground-up data platform builds.
• Strong Python skills for data transformation and orchestration.
• Deep understanding of modern data architectures, pipelines, and cloud-native solutions - especially AWS.
• Able to operate strategically (architecture, direction-setting) and tactically (hands-on engineering).
• Active SC Clearance (must be current).