Senior Data Engineer

FPSG; Glasgow; On-site; GBP 65,000 - 90,000; Full time

Job summary

FPSG is seeking a Senior Data Engineer to own and optimize a Databricks-based data platform. You will write pipeline code, build jobs, and model data as a core daily task, shaping the data layer for Fabric/OneLake.

You will design, deploy, and maintain pipelines, ensure robust data architectures, and apply Delta Lake patterns with a strong emphasis on SQL and Python/Scala for pipeline logic.

Qualifications

• Hands-on experience building and running Databricks Pipelines/Jobs.
• Strong data modelling ability from raw data to silver/gold tables.
• Solid Delta Lake knowledge including time travel and optimization.
• Experience with Databricks Asset Bundles or equivalent deployment/CI practices.
• Practical experience building environment-variable-driven Azure Data Factory pipelines.

Responsibilities

• Design, build, and operate Databricks Pipelines and Jobs.
• Design and build Delta Lake pipelines.
• Own Databricks deployment bundles, packaging, versioning, and promotion of pipelines.
• Build data pipelines using environment-variable-driven Azure Data Factory pipelines.
• Contribute to the data layer feeding Microsoft Fabric/OneLake.

Skills

Databricks pipelines

Delta Lake

Azure Data Factory

SQL

Python/Scala

Data modelling

Data architecture

CI/CD pipelines

Tools

Databricks Asset Bundles

Job description

Exciting opportunity for a Senior Data Engineer to own the maintenance, expansion, and performance of Databricks-based data platform. This is a hands-on role, you'll be writing pipeline code, building jobs, and modelling as the main part of your day to day.

Role Responsibilities

Responsibilities will include:

• Designing, building, and operating Databricks Pipelines and Jobs
• Designing and building Delta Lake pipelines
• Owning Databricks deployment bundles, packaging, versioning, and promoting pipeline code and job definitions
• Building and maintaining environment-variable-driven Azure Data Factory pipelines
• Building the data layer that feeds Microsoft Fabric/OneLake

Person Specification

Essential skills will include:

• Hands-on experience building and running Databricks Pipeline/Job structure in a production
• Strong data modelling ability, able to look at raw source data and design sensible, well-structured silver/gold tables
• Strong understanding of medallion (bronze/silver/gold) design patterns, and data architecture right at table level
• Solid Delta Lake knowledge, deletion vectors, time travel, optimization, compatibility considerations
• Experience with Databricks Asset Bundles or equivalent deployment/CI practices for data pipelines
• Practical experience building environment-variable/parameter-driven Azure Data Factory pipelines
• Proven experience of managing large data architectures
• Strong SQL, and comfort in at least one of Python/Scala for pipeline logic

You will be the technical expert for the data solution, to work alongside the engineering lead to manage the data aspects of the digital solution.

FPSG is committed to equal opportunities regardless of gender, race, disability, sexual orientation, religion or belief and age.