Data Engineer

Talantir; Greater Manchester; Hybrid; GBP 54.000 - 66.000; Vollzeit

Benefits dieser Stelle

Discretionary bonus

Share option plan

Pension scheme

Life insurance

Health plan

Holiday entitlement

Study support

Employee assistance 24/7

Office social events

Zusammenfassung

Talantir is hiring a Data Engineer to design and optimise scalable data pipelines in a Databricks Lakehouse. You will work with streaming and batch data using Spark Structured Streaming and Kafka (MSK), ensuring efficient data flows and governance across analytics, reporting, and operations.

Your role includes CI/CD for data workflows and close collaboration with BI, product, and engineering teams. The position offers a hybrid working model in a collaborative office environment with a

Qualifikationen

• Strong experience with Databricks and Spark (PySpark or Scala).
• Proven capability building streaming and batch pipelines using Spark Structured Streaming and Kafka (MSK).
• Solid AWS experience (S3, Glue, Lambda, Batch, IAM) and data modelling skills.

Aufgaben

• Design, build, and optimise scalable data pipelines within a Databricks Lakehouse environment.
• Develop streaming and batch pipelines using Spark Structured Streaming and Kafka; ensure data quality and governance.
• Collaborate with BI, product, and engineering to deliver high-quality, governed data for analytics and reporting.

Kenntnisse

Databricks

Apache Spark

Python

SQL

PySpark

Scala

CI/CD

Git

AWS

Delta Lake

Delta Live Tables

Medallion Architecture

Spark Structured Streaming

Kafka

MSK

S3

Glue

Lambda

Batch processing

IAM

Tools

Databricks

Apache Spark

Kafka (MSK)

Delta Lake

Delta Live Tables

AWS

Python

SQL

CI/CD tools

Git

Jobbeschreibung

Data Engineer

Salary up to £60,000 + discretionary bonus

Hybrid working model with collaborative office environment

Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS

ABOUT THE CLIENT

Due to continued growth, we’re seeking a skilled Data Engineer to join an established organisation at the forefront of building modern, data-driven platforms. The business is investing heavily in its data ecosystem and is focused on delivering real-time insights and trusted analytics that support teams across product, finance, marketing, and compliance. This is an excellent opportunity to work with modern cloud data technologies and play a key role in shaping a scalable, secure, and high-performance data platform.

WHAT WILL YOU BE DOING?

As a Data Engineer, you will design, build, and optimise scalable data pipelines within a Databricks Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering teams, you will help deliver high-quality, governed data that supports analytics, reporting, and operational decision-making. You’ll also play a key role in enabling business intelligence through high-performance data models and integrations with reporting tools. You’ll monitor and optimise pipeline performance, implement CI/CD processes for data workflows, and maintain strong governance and security standards across the data platform. Documentation, observability, and cost optimisation will also form part of your responsibilities as you contribute to the ongoing development of the organisation’s data ecosystem.

OUR BENEFITS:

• Competitive basic salary
• Discretionary bonus scheme
• Company shares option plan
• Contributory pension scheme
• Life insurance (4x basic salary)
• Health cash plan
• Generous holiday allowance including bank holidays
• Study support and clear progression opportunities
• Employee assistance programme available 24/7
• Collaborative office environment with regular social and charity events
• And Much More!!!

DATA ENGINEER - ESSTENTIAL SKILLS

• Strong experience with Databricks and Apache Spark (PySpark or Scala)
• Experience building streaming and batch pipelines using Spark Structured Streaming
• Hands-on experience with Kafka (MSK) and real-time data ingestion
• Strong understanding of Delta Lake, Delta Live Tables, and Medallion Architecture
• Solid AWS experience including services such as S3, Glue, Lambda, Batch, and IAM
• Proficiency in Python and SQL for data engineering and analytics
• Experience implementing CI/CD pipelines (GitHub Actions, Jenkins, or Azure DevOps)
• Strong Git and version control practices
• Experience tuning and optimising Spark workloads

KEY SKILLS

Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data