Real-Time Data Engineer - Databricks Lakehouse Expert

UK Tote Group

Greater Manchester

On-site

GBP 65,000 - 90,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Competitive Basic Salary
Discretionary Bonus Scheme
Company Shares Option Plan
Contributory pension scheme
Life insurance (4 x basic salary)
Simply Health Cash Plan
Holiday entitlement (33 days incl. BH)
Study Support and progression
24/7 employee assistance helpline
Agile office environment with free par
Regular social and volunteering events

Job summary

UK Tote Group is seeking an experienced Data Engineer to design, build and optimise a Databricks-based Lakehouse enabling real-time data, analytics and reporting across the Tote. You will work onsite at the Wigan Office at least one day per week, collaborating with engineering and BI teams to deliver scalable, secure data pipelines on AWS using Spark, Kafka and Delta Lake.

Join a fast-paced, collaborative team and contribute to the Medallion architecture, data ingestion from S3, and the shaping

Qualifications

  • Experience building pipelines in Databricks with Spark (PySpark) and Structured Streaming.
  • Strong knowledge of Delta Lake, DLT, and Medallion Architecture.
  • Solid AWS experience, especially S3, plus monitoring tools.
  • Proficient in Python and SQL for data engineering and analytics.

Responsibilities

  • Design, build, and optimise streaming and batch pipelines in Databricks.
  • Develop bronze, silver, and gold delta tables for downstream analytics and APIs.
  • Ingest data from AWS MSK (Kafka), S3, and external APIs into the Lakehouse.
  • Collaborate with reporting engineers to enable high-performance dashboards.
  • Monitor and tune pipeline performance using Databricks metrics and CloudWatch.

Skills

Python
SQL
CI/CD pipelines
Analytical thinking
Strong communication

Tools

Databricks
Apache Spark
Kafka (MSK)
Delta Lake
Declarative Pipelines (DLT)
Medallion Architecture
AWS (S3)
CloudWatch
Databricks SQL Warehouses
GitHub Actions
Azure DevOps

Job description

UK Tote Group is seeking an experienced Data Engineer to design, build and optimise a Databricks-based Lakehouse enabling real-time data, analytics and reporting across the Tote. You will work onsite at the Wigan Office at least one day per week, collaborating with engineering and BI teams to deliver scalable, secure data pipelines on AWS using Spark, Kafka and Delta Lake.

Join a fast-paced, collaborative team and contribute to the Medallion architecture, data ingestion from S3, and the shaping

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

UK Tote Group • Wigan

On-site
GBP 70,000 - 90,000
Competitive Basic Salary
Discretionary Bonus Scheme
Company Shares Option Plan
+5
Data Engineer
Data Engineer

UK Tote Group • Greater Manchester

On-site
GBP 65,000 - 90,000
Competitive Basic Salary
Discretionary Bonus Scheme
Company Shares Option Plan
+8
Databricks Data Engineer - Lakehouse & AI Enablement
Databricks Data Engineer - Lakehouse & AI Enablement

NTT DATA • Manor Park

On-site
GBP 60,000 - 90,000
Databricks Data Engineer – Lakehouse & AI Enablement
Databricks Data Engineer – Lakehouse & AI Enablement

NTT DATA, Inc. • Greater London

On-site
GBP 70,000 - 110,000
Databricks Lakehouse Engineer - Hybrid (London/Manchester)
Databricks Lakehouse Engineer - Hybrid (London/Manchester)

Gazelle Global • City Of London

Hybrid
GBP 85,000 - 120,000
Senior Data Engineer – Databricks Lakehouse (Hybrid)
Senior Data Engineer – Databricks Lakehouse (Hybrid)

La Fosse Associates • Milton Keynes

Hybrid
GBP 92,000 - 101,000
Databricks Engineer
Databricks Engineer

Gazelle Global • City Of London

Hybrid
GBP 85,000 - 120,000
Databricks Consultant & Lakehouse Data Engineer (Remote)
Databricks Consultant & Lakehouse Data Engineer (Remote)

Consult • Leeds

Hybrid
GBP 83,000 - 166,000
Senior Data Engineer — Databricks Lakehouse (Hybrid, MK)
Senior Data Engineer — Databricks Lakehouse (Hybrid, MK)

Data & AI • Milton Keynes

Hybrid
GBP 92,000 - 101,000
Databricks Data Engineer: Scalable Lakehouse Pipelines
Databricks Data Engineer: Scalable Lakehouse Pipelines

Pulse Group • Reading

On-site
GBP 63,000 - 77,000