Senior Data Engineer (Data, AI, ML, Data Science)

Switch Tech Talent; Newcastle upon Tyne; On-site; GBP 117,000 - 143,000; Full time

Benefits offered by this job

Private healthcare

Pension

25 days holiday

Job summary

Switch Tech Talent in Newcastle upon Tyne is seeking a Senior Data Engineer to design, build and maintain scalable data pipelines and data-intensive applications supporting analytics, operational systems and AI/ML use cases.

You will work across AI engineering, machine learning and data science, using Python, SQL, Spark, Databricks and Airflow, taking ownership from day one and collaborating with end users to shape technical solutions and deliveries.

Qualifications

• Strong Python, SQL and data engineering fundamentals.
• Experience with distributed processing and workflow orchestration (Spark, Databricks or Airflow).
• Practical understanding of data quality, performance and reliability trade-offs.
• Clear communication and ability to own unfamiliar problems.

Responsibilities

• Design, build and maintain data pipelines and data-intensive applications.
• Develop ETL/ELT pipelines to ingest, transform and validate data.
• Build maintainable Python and SQL solutions and scalable data workflows.
• Work with Spark, Databricks and Airflow; improve data quality and performance.
• Collaborate with stakeholders to deliver from design to production.

Skills

Python

SQL

Data pipelines

Data engineering fundamentals

Distributed processing

Airflow

Spark

Databricks

Communication

Ownership

Education

2:1 STEM degree

Tools

Spark

Databricks

Airflow

Job description

Newcastle upon Tyne | On-site

Financial Services or National Security | Separate divisions

A growing technology vendor is hiring Senior Data Engineers to build scalable data platforms and pipelines, with opportunities to work across AI engineering, machine learning and data science.

This is a hands-on role for a Senior Data Engineer who enjoys solving complex problems and wants to work beyond a narrowly defined pipeline-development remit. Strong Python, SQL and data engineering fundamentals are the starting point, alongside the curiosity to explore how data, models and software can solve real business problems.

They deliver predominantly greenfield projects through separate financial services and national security divisions, with further expansion into insurance and energy planned. Senior Data Engineers work closely with end users, take ownership from day one and help shape both the technical approach and the finished solution.

The Senior Data Engineer role

As the Senior Data Engineer, you will design, build and maintain reliable data pipelines and data-intensive applications supporting analytics, operational systems and AI/ML use cases.

• Developing ETL/ELT pipelines to ingest, transform and validate data, making it accessible for downstream analysis and modelling.
• Building maintainable Python and SQL solutions, from reusable processing components to scalable data workflows.
• Working with distributed data processing technologies such as Apache Spark and Databricks, alongside orchestration tools such as Apache Airflow.
• Improving data quality, reliability and performance, investigating issues and addressing their underlying causes.
• Working directly with stakeholders and end users to understand requirements and take technical solutions from design through to delivery.

Opportunities across AI, machine learning and data science

The role offers scope to contribute beyond core data engineering, supporting analytical and modelling work and integrating AI components into applications and workflows.

Depending on the project, this could include RAG, LLM integration, model evaluation, inference optimisation, and testing and debugging non-deterministic systems.

The team values sound engineering judgement: understanding model limitations, identifying failure modes and recognising when statistical methods or deterministic software are more appropriate than AI.

You do not need to be a specialist in every area.

The priority is strong data engineering capability and a genuine willingness to broaden your contribution across data science, machine learning and applied AI.

Senior Data Engineer experience and skills

• Strong commercial experience with Python, SQL, data pipelines and data-intensive systems.
• Experience with distributed processing and workflow orchestration, with tools such as Spark, Databricks or Airflow.
• A practical understanding of how systems work, including data quality, performance, reliability and technical trade-offs.
• Clear communication, a collaborative approach and the ability to take ownership of unfamiliar problems.
• A 2:1 or above in a STEM subject from a leading university.

Experience with LLMs, transformer architectures, AI evaluation or explainability would be beneficial.

Exposure to edge computing, constrained or air-gapped environments, or AI safety and security would also be valuable.

Financial services, banking, private capital, risk, front-office technology, defence or other regulated-sector experience is advantageous, but not essential.

Why join as a Senior Data Engineer?

The opportunity combines greenfield development, direct access to end users and meaningful technical ownership within a collaborative engineering environment. It would suit a Senior Data Engineer who enjoys variety, wants to understand the wider purpose of their work and is keen to develop across data platforms, AI/ML and data science.

The package includes a salary of up to £130,000, a target-based bonus of approximately 10-25%, private healthcare, pension, and 25 days' holiday plus bank holidays.