Senior Data Engineer: Databricks Platform Lead

FPSG; Glasgow; On-site; GBP 75,000 - 110,000; Full time

Job summary

FPSG in Glasgow seeks a Senior Data Engineer to own the Databricks-based data platform's maintenance, expansion, and performance. This hands-on role centers on writing pipeline code, building jobs, and data modelling as the daily core tasks.

You design and operate pipelines, collaborate with source systems, and implement Delta Lake pipelines while controlling deployment bundles and CI practices.

Qualifications

• Strong SQL and data modelling skills.
• Hands-on experience with Azure data platforms in production.
• Experience building Databricks Pipelines and Jobs in production.
• Background with Snowflake and interest in Databricks.
• Ability to design silver/gold tables and medallion patterns.

Responsibilities

• Designing, building, and operating Databricks Pipelines and Jobs.
• Designing and building Delta Lake pipelines.
• Owning Databricks deployment bundles, packaging, versioning, and promoting code.
• Building environment-variable-driven Azure Data Factory pipelines.
• Building the data layer that feeds Microsoft Fabric/OneLake.
• Monitoring and troubleshooting data pipelines for performance, reliability and cost.
• Taking ownership of data solutions from source systems through to business outcomes.

Skills

SQL

Python

Scala

Data modelling

Ownership mindset

Tools

Databricks Pipelines & Jobs

Delta Lake

Azure Data Factory

Databricks Asset Bundles

Snowflake

Job description

FPSG in Glasgow seeks a Senior Data Engineer to own the Databricks-based data platform's maintenance, expansion, and performance. This hands-on role centers on writing pipeline code, building jobs, and data modelling as the daily core tasks.

You design and operate pipelines, collaborate with source systems, and implement Delta Lake pipelines while controlling deployment bundles and CI practices.