Senior Data Engineering Consultant (Databricks)

Talan Group; Greater London; Hybrid; GBP 90,000 - 120,000; Full time

Benefits offered by this job

25 days holiday + bank holidays

Private medical insurance

Life cover

Cycle to work scheme

Pension scheme with employer contrib.

Employee assistance programme

Udemy for Business learning

Job summary

Talan Data x AI is a leading data management and analytics consultancy seeking a Senior Data Engineering Consultant (Databricks) to join on client-site delivery in London. You will design and optimise data pipelines with PySpark in Databricks, build dashboards in Power BI, and lead technical and project teams while engaging stakeholders.

The role requires multi-disciplinary data expertise, strong communication, and a proactive, adaptable mindset, with willingness to travel.

Qualifications

• Multiskilled experience across data management, data engineering, warehousing, modelling, quality, integration, analytics, visualization, data science and BI.
• Databricks with PySpark; build and optimise data pipelines in Databricks environments.
• Proven experience designing dashboards/reports with Power BI.
• Strong Python and SQL skills; ability to write complex queries.
• Experience with Tableau, Power BI, Cloud (Azure/AWS/GCP) and Databricks integrations.
• Experience with data ingestion, transformation and quality control processes.
• Excellent problem solving and ability to work under pressure.
• Strong communication and stakeholder management skills.
• Experience leading technical/project teams.
• Awareness of industry standards, regulations, and developments.
• End-to-end project delivery with various development methodologies (Agile, Waterfall, Scrum, DevOps).
• DevOps and infrastructure deployments (Azure and Databricks).
• Ideally SAS and relational data warehousing concepts; enterprise ETL tools like Informatica, Talend, Datastage or Alteryx.
• Cross and multi-platform experience; team building and leadership.

Skills

Databricks

Python

SQL

Power BI

Tableau

Azure

AWS

GCP

Snowflake

Data Modeling

Data Integration

Tools

Informatica

Talend

Datastage

Alteryx

Job description

Senior Data Engineering Consultant (Databricks)

Senior Data Engineering Consultant (Databricks)

• Strong team player with the ability to work independently
• Excellent client-facing and stakeholder engagement skills
• Proactive, self-motivated, and adaptable
• Strong verbal, written, and presentation skills
• Ability to build effective internal and external relationships
• Strong influencing and negotiation skills
• Creative thinker with the ability to propose innovative solutions
• Passion for continuous learning and self-development
• Leadership capability, either formally or through delivery ownership

To qualify for this role, you must have:

• Multi-skilled experience in one or more of the following disciplines: Data Management, Data Engineering, Data Warehousing, Data Modelling, Data Quality, Data Integration, Data Analytics, Data Visualisation, Data Science and Business Intelligence.
• Confident in delivering solutions using Databricks & developing and optimising data pipelines using PySpark within Databricks environments.
• Proven experience building, publishing, and supporting dashboards and reports using Power BI
• Strong experience and programming skills in languages such as Python and SQL and the ability to write complex SQL queries.
• Project experience using one or more of the following technologies: Tableau, Power BI, Cloud, Azure, AWS, GCP, Snowflake) and their integration with Databricks.
• Experience with data ingestion, transformation, and quality control processes.
• Excellent problem-solving skills and the ability to think critically under pressure.
• Strong communication and interpersonal skills, with the ability to work effectively with clients and stakeholders.
• Experience of leading technical and/or project teams.
• A proactive awareness of industry standards, regulations, and developments.
• Project lifecycle experience, having played a leading role in the delivery of end-to-end projects, as well as a familiarity with different development methodologies and disciplines (e.g. Agile, Waterfall Scrum, DevOps, Testing).
• Experience of DevOps and infrastructure deployments (Azure and Databricks).

Ideally, you’ll also have:

• Delivery experience using the following technology: SAS (SAS EG, SAS DI, SAS Viya).
• Relational Databases and Data Warehousing concepts.
• Enterprise ETL tools such as Informatica, Talend, Datastage or Alteryx.
• Project experience using the any of the following technologies: Hadoop, Spark, Scala, Oracle, Pega, Salesforce.
• Cross and multi-platform experience.
• Team building and leading.

#TalanUK#LI-HB1

Qualifications

You must be:

• Willing to work on client sites, potentially for extended periods.
• Willing to travel for work purposes and be happy to stay away from home for extended periods.
• Eligible to work in the UK without restriction.

Additional Information

What we offer:

• BDP Plus - A reward programme whereby you accrue points to trade against a 3-month paid sabbatical or cash equivalent.
• 25 days holiday + bank holidays.
• 5 days holiday buy/sell option.
• Private medical insurance.
• Life cover.
• Cycle to work scheme.
• Eligibility for company pension scheme (5% employer contribution, salary sacrifice option).
• Employee assistance programme.
• Bespoke online learning via Udemy for Business.

Company Description

Talan Data x AI is a leading Data Management and Analytics consultancy, working closely with leading software vendors and top industry experts across a range of sectors, unlocking value and insight from their data. At Talan Data x AI, innovation is at the heart of our client offerings, and we help companies to further improve their efficiency with modern processes and technologies, such as Machine Learning (ML) and Artificial Intelligence (AI).

Our consultants are at the heart of everything we do, and we have been recertified as a 2025 Great Place to Work®. This achievement not only highlights Talan Data x AI’s positive organisational culture but also strengthens its reputation as an employer of choice within the industry. We invest heavily in the training and development of our teams and hold regular socials in each region to encourage engagement and network building.

Senior Data Engineering Consultant (Databricks)