Senior Data Engineering Consultant (Databricks)

Talan; Greater London; On-site; GBP 70,000 - 100,000; Full time

Benefits offered by this job

BDP Plus – A reward programme whereby

25 days holiday + bank holidays.

5 days holiday buy/sell option.

Private medical insurance.

Life cover.

Cycle to work scheme.

Eligibility for company pension scheme

Employee assistance programme.

Bespoke online learning via Udemy for

Job summary

Talan Data x AI in London is seeking a Senior Data Engineering Consultant (Databricks) to lead data integration and analytics projects for enterprise clients. The role requires deep Databricks, PySpark, SQL and Power BI skills, plus strong stakeholder engagement and leadership.

You will design pipelines, build dashboards, mentor delivery teams in Agile environments, and communicate clearly with technical and non-technical audiences.

Qualifications

• Deep experience with Databricks and building data pipelines.
• Proficient in PySpark and SQL; able to optimize complex queries.
• Strong dashboarding with Power BI and data visualization tools.

Responsibilities

• Lead end-to-end data integration and analytics projects for clients.
• Design, develop, and optimize data pipelines; mentor delivery teams.
• Engage with stakeholders and communicate technical concepts effectively.

Skills

Databricks

PySpark

Power BI

SQL

Python

Data Engineering

Client-facing

Leadership

Tools

Databricks

Azure

AWS

GCP

Tableau

Job description

Senior Data Engineering Consultant (Databricks)

• Full-time
• Contract Type: Permanent contract

Talan Data x AI is a leading Data Management and Analytics consultancy, working closely with leading software vendors and top industry experts across a range of sectors, unlocking value and insight from their data. At Talan Data x AI, innovation is at the heart of our client offerings, and we help companies to further improve their efficiency with modern processes and technologies, such as Machine Learning (ML) and Artificial Intelligence (AI).

Our consultants are at the heart of everything we do, and we have been recertified as a 2025 Great Place to Work®. This achievement not only highlights Talan Data x AI’s positive organisational culture but also strengthens its reputation as an employer of choice within the industry. We invest heavily in the training and development of our teams and hold regular socials in each region to encourage engagement and network building.

Your skills and attributes for success:

• Strong team player with the ability to work independently
• Excellent client‑facing and stakeholder engagement skills
• Proactive, self‑motivated, and adaptable
• Strong verbal, written, and presentation skills
• Ability to build effective internal and external relationships
• Strong influencing and negotiation skills
• Creative thinker with the ability to propose innovative solutions
• Passion for continuous learning and self‑development
• Leadership capability, either formally or through delivery ownership

To qualify for this role, you must have:

• Multi-skilled experience in one or more of the following disciplines: Data Management, Data Engineering, Data Warehousing, Data Modelling, Data Quality, Data Integration, Data Analytics, Data Visualisation, Data Science and Business Intelligence.
• Confident in delivering solutions using Databricks & developing and optimising data pipelines using PySpark within Databricks environments.
• Proven experience building, publishing, and supporting dashboards and reports using Power BI
• Strong experience and programming skills in languages such as Python and SQL and the ability to write complex SQL queries.
• Project experience using one or more of the following technologies: Tableau, Power BI, Cloud, Azure, AWS, GCP, Snowflake) and their integration with Databricks.
• Experience with data ingestion, transformation, and quality control processes.
• Excellent problem-solving skills and the ability to think critically under pressure.
• Strong communication and interpersonal skills, with the ability to work effectively with clients and stakeholders.
• Experience of leading technical and/or project teams.
• A proactive awareness of industry standards, regulations, and developments.
• Project lifecycle experience, having played a leading role in the delivery of end-to-end projects, as well as a familiarity with different development methodologies and disciplines (e.g. Agile, Waterfall Scrum, DevOps, Testing).
• Experience of DevOps and infrastructure deployments (Azure and Databricks).

Ideally, you’ll also have:

• Delivery experience using the following technology: SAS (SAS EG, SAS DI, SAS Viya).
• Relational Databases and Data Warehousing concepts.
• Enterprise ETL tools such as Informatica, Talend, Datastage or Alteryx.
• Project experience using the any of the following technologies: Hadoop, Spark, Scala, Oracle, Pega, Salesforce.
• Cross and multi-platform experience.
• Team building and leading.

You must be:

• Willing to work on client sites, potentially for extended periods.
• Willing to travel for work purposes and be happy to stay away from home for extended periods.
• Eligible to work in the UK without restriction.

What we offer:

• BDP Plus – A reward programme whereby you accrue points to trade against a 3-month paid sabbatical or cash equivalent.
• 25 days holiday + bank holidays.
• 5 days holiday buy/sell option.
• Private medical insurance.
• Life cover.
• Cycle to work scheme.
• Eligibility for company pension scheme (5% employer contribution, salary sacrifice option).
• Employee assistance programme.
• Bespoke online learning via Udemy for Business.