Senior Data Engineer

C&W Services (S); Greater London; Remote; GBP 90,000 - 130,000; Full time

Job summary

C&W Services (S) seeks a Senior Data Engineer to drive data transformation initiatives across EMEA and APAC. You will shape the Databricks-based Lakehouse, design scalable data models, and implement agentic AI capabilities at enterprise scale.

In this hands-on role, you will work with a team of analytical engineers, make architectural decisions, and lead production data pipelines. This is a senior role demanding strategic thinking and deep technical expertise in modern data platforms.

Qualifications

• Experience designing scalable data platforms.
• Experience with Databricks-based pipelines and governance.

Responsibilities

• Lead data transformation projects across EMEA/APAC.
• Architect data models and pipelines for enterprise-scale use.
• Troubleshoot production data issues and perform root-cause analysis.
• Collaborate with engineers and analysts to improve data quality.

Skills

Systems thinking

Production troubleshooting

Data modeling

Advanced SQL

Python

PySpark

Education

Degree in CS, data science, or related field

Tools

Databricks

Azure

Unity Catalog

Job description

Job Title

Senior Data Engineer

Job Description Summary

Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC

We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a Senior Data Engineer to help build what comes next.

Our Databricks-native Lakehouse is established and mature and already supports the business end to end. The work ahead is more ambitious: applying agentic capabilities at genuine enterprise scale, across a breadth of domains, and shaping how we win in the age of AI.

Job Description

Role: Senior Data Engineer

Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC

We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a Senior Data Engineer to help build what comes next.

Our Databricks-native Lakehouse is established and mature and already supports the business end to end. The work ahead is more ambitious: applying agentic capabilities at genuine enterprise scale, across a breadth of domains, and shaping how we win in the age of AI.

This is a hands-on senior role on key transformation projects across EMEA and APAC. You will work directly with the team lead and a group of analytical engineers who push each other to stay at the frontier of what is possible.

You will take on problems without a documented precedent, make architectural decisions of real consequence, and build depth across multiple domains early enough to influence where the platform goes next. Very few organisations in our sector have started this work, so the skill set you build here is not one you could assemble elsewhere in this industry.

What you will work on

• Driving the shift from a code writing team to an AI orchestration team. This means building agentic workflows that handle data engineering tasks and CI/CD, designing semantic models for downstream consumption, and implementing an AI Gateway for governance.
• Stewarding our AI amplified Databricks based framework, maintaining it and improving it incrementally as we learn.
• Designing data models that bring sources from anywhere in the world into a single coherent shape.
• Building data solutions for domains like custom MDM, geocoding, client mastering, and profiling. Real estate context is a plus but not required .
• Designing and implementing stable, automated ingestion pipelines from diverse sources.
• Owning production troubleshooting and root cause analysis. When things go wrong, you diagnose and resolve pipeline failures, performance issues, and data quality problems by working through code, logs, and documentation systematically.
• Reporting on data quality and shipping the improvements that move the numbers.

What we are looking for

Core technical skills

• Systems thinking. You see the bigger picture and build for scale rather than for the ticket in front of you.
• Production troubleshooting and root cause analysis. You diagnose and resolve pipeline failures, performance issues, and data quality problems under pressure, working through code, logs, and documentation systematically rather than guessing.
• Data modeling. You design cross functional data products, establish data contracts, and handle complex business rules.
• Advanced SQL, including window functions, query optimization, and MERGE/UPSERT operations.
• Python and PySpark. You write reusable, parameterized functions and work comfortably with JSON, CSV, and Parquet.

Platform knowledge

• Deep experience with Databricks (Declarative Pipelines, DABS, Delta Lake, Spark optimization, job orchestration).
• Familiarity with the Azure ecosystem.
• Working understanding of Unity Catalog.

About you

• Around 6 or more years of relevant experience.
• A degree in a quantitatively rigorous field such as computer science, data science, econometrics, mathematics, or physics.
• Driven and curious. You want to work on things that have not been figured out yet.
• Strong written communication. Async updates and clear documentation are part of the job.

Nice to have

• Experience with agentic data engineering , using coding agents in real delivery and building the harness around them: repo conventions, reusable context, tests and guardrails that make agent output trustworthy.
• Geospatial data processing (H3 indexes, spatial SQL, point in polygon at scale).
• Recursive CTEs and complex SQL patterns.
• Infrastructure familiarity (Azure Portal, resource management, CLI).
• Strong Git workflows and code review habits.
• Real estate domain experience.

What we offer

• Autonomy. We set the high level roadmap. You decide how to get there.
• Dedicated learning time and a team that levels each other up. The frontier moves weekly. We give you the time to keep up with it, and you will be surrounded by people doing the same.
• A rare growth path. You will build leading capability in a field where very few organisations in our sector have yet begun, giving you a skill set you could not have assembled elsewhere in this industry.
• A solid techni