Senior Data Engineer - Databricks

hireful; City Of London; Remote; GBP 77,000 - 94,000; Full time

Benefits offered by this job

Health insurance

Bonus 10%

Job summary

hireful is seeking a Senior Data Engineer to shape a scalable data platform, mainly using Databricks. You will own data pipelines, quality, governance, and access controls, while mentoring a junior engineer.

Location is UK-based remote with occasional London visits. The role offers up to £85,000 base plus a 10% bonus and extensive benefits.

Qualifications

• Senior level experience in data engineering with hands-on Databricks skills.
• Advanced SQL and Python coding abilities.
• Track record building enterprise-scale data pipelines and ETL/ELT solutions.
• Strong grounding in data architecture, modelling and platform design.
• Experience mentoring junior data engineers.
• Experience with diverse data sources and cloud environments (AWS/Azure).
• Desirable: interest in ML/AI and predictive modelling.

Responsibilities

• Design, build and maintain scalable data pipelines and products using Databricks.
• Develop and optimise ETL/ELT processes ingesting data from internal and external sources.
• Shape enterprise data architecture balancing scalability, performance and security.
• Partner with Finance, Product, Sales, Marketing and Engineering to translate business needs into solutions.
• Embed data quality, governance and access controls across the data lifecycle.
• Mentor a junior data engineer and help mature the data function.

Skills

Databricks

PySpark

Spark SQL

Delta Lake

SQL

Python

ETL/ELT

Data architecture

Mentoring

AWS / Azure

CI/CD

Job description

UK HQ based & fast-growing software business (400+ staff, globally), serving hundreds of enterprise customers worldwide, are looking for a Senior Data Engineer to help shape the future of its data platform. You'll take ownership of building scalable data solutions in Databricks primarily, establishing a single source of truth, and enabling trusted, self-service access to data across the business.

Location: UK, remote with very occasional visits to the office in London. Salary: Up to £85,000 basic, plus 10% bonus and an extensive benefits package.

What you'll be doing

• Design, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake.
• Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems.
• Shape enterprise data architecture, balancing scalability, performance, security and maintainability.
• Partner with stakeholders across Finance, Product, Sales, Marketing and Engineering to turn business challenges into technical solutions.
• Embed data quality, governance and access controls throughout the data lifecycle.
• Mentor a junior engineer and help mature a growing data function.

What you need to bring

• “Senior” level experience in data engineering, with strong hands-onDatabricks skills (PySpark, Spark SQL, Delta Lake, etc)
• Advanced SQL and Python coding skills.
• A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions.
• Solid grounding in data architecture, modelling and modern data platform design.
• Experience of mentoring more junior Data Enigneers
• A broad range of data source experience (structured & unstructured): from CRM / revenue data, to financial data & subscription usage data.
• Cloud experience, preferably AWS (but Azure is totally fine), and familiarity with CI/CD and source control best practice.
• Desirable: More Data Science leaning background or want to explore in this role:ML / AI & data / predictive modelling strategies.

Why you'll love this job

• A highly visible, hands-on role with real influence over data architecture and platform strategy.
• Join a small, high-impact team at the heart of the company's data strategy, with genuine scope to shape how it evolves. We are near the beginning of our Data Ops’ journey – so an excellent opportunity to shape the future!
• Extensive benefits package including health insurance, income protection, life assurance, subsidised gym membership and 25 days' holiday plus your birthday off.

If you're a Databricks specialist who wants to build something from the ground up rather than maintain someone else's legacy, we'd love to hear from you.