Senior Data Engineer: Databricks & Snowflake Lead

Novatus; Greater London; On-site; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Private Medical Insurance

Life Insurance

Employee Assistance Programme

Flexible hours

Gym & fitness discounts

Office snacks

Job summary

Novatus is seeking a Senior Data Engineer to build and scale our Databricks-based data platform and Snowflake data warehouse for regulatory reporting workloads. You will write clean, tested code, own data pipelines, and provide technical leadership across the team.

You will mentor junior engineers, design auditable pipelines in PySpark and Python, optimize Spark and Snowflake performance, and collaborate with product, compliance, and DevOps to deliver reliable, compliant data solutions in a

Qualifications

• 8+ years as a software engineer with 5+ years in data engineering.
• Experience designing auditable, reproducible data pipelines in regulated environments.
• Able to write and optimize complex SQL queries on large data sets.
• Strong data modeling and warehouse design fundamentals.
• Strong software engineering fundamentals: clean code, DRY, SOLID, testing, CI/CD, observability.
• Deep experience with a cloud lakehouse / data warehouse (Databricks, Snowflake, BigQuery, Redshift, EMR).
• Experience with a workflow orchestrator (Airflow, Dagster, or Databricks Workflows).
• Able to own architectural decisions and deliver in incremental, shippable slices.

Responsibilities

• Design and evolve our Databricks-based data framework and Snowflake warehouse.
• Coach and mentor junior engineers, and lead technical discussions and design reviews.
• Design, build, and optimize data pipelines using Databricks, Python and PySpark, with event-driven components (e.g. Kafka).
• Ensure pipelines are auditable, lineage-aware, idempotent, and resilient in regulated environments.
• Implement robust data quality controls including testing, validation, monitoring, and alerting.
• Drive performance optimization across Spark and Snowflake workloads.
• Partner with Product, Engineering, DevOps, and Regulatory teams to translate requirements into scalable technical designs.
• Contribute to the continuous improvement of our development processes and tools.

Skills

Data engineering

Python

PySpark

SQL

Databricks

Snowflake

Cloud data warehousing

CI/CD

Leadership

Airflow

Dagster

Databricks Workflows

Tools

Airflow

Dagster

Databricks Workflows

Job description

Novatus is seeking a Senior Data Engineer to build and scale our Databricks-based data platform and Snowflake data warehouse for regulatory reporting workloads. You will write clean, tested code, own data pipelines, and provide technical leadership across the team.

You will mentor junior engineers, design auditable pipelines in PySpark and Python, optimize Spark and Snowflake performance, and collaborate with product, compliance, and DevOps to deliver reliable, compliant data solutions in a