Azure Databricks Technical Lead

Capgemini; Manchester; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

Capgemini is seeking an Azure Databricks Engineer to join our ACT Practice in the UK, delivering scalable data engineering solutions. You will implement Databricks platforms, optimize workloads, and partner with Data Engineers and Data Scientists to drive value.

Strong Python, PySpark, and SQL skills are essential, along with REST API integration and cost-aware operations. Hybrid working spans offices, client sites, and home, with comprehensive pre-employment checks.

Qualifications

• Hands-on implementation and troubleshooting of Azure Databricks.
• Design and build ingestion and transformation pipelines using PySpark and SQL.
• Secure REST API integration with databases and SaaS apps.
• Cost-aware platform operations and monitoring in Azure.

Skills

Python

PySpark

SQL

REST API integration

Agile delivery

Collaboration

Data pipelines design

Monitoring & incident management

Tools

Azure Databricks

Databricks Serverless

Delta Lake

BI tooling integration

RStudio migration

Job description

About The Job You're Considering

We work closely with clients and partners to take full advantage of the opportunities of technology. We mobilize expert teams that create custom solutions from existing and emerging technology to deliver viable outcomes at speed. We call this 'Value in the making.'

Internal Role Information

Grade – C1

Location -Anywhere in the UK

About The Job You're Considering

We work closely with clients and partners to take full advantage of the opportunities of technology. We mobilize expert teams that create custom solutions from existing and emerging technology to deliver viable outcomes at speed. We call this 'Value in the making.'

Part of the Capgemini Group, Sogeti makes business value through technology for organizations that need to implement innovation at speed and want a local partner with global scale. With a hands‑on culture and close proximity to its clients, Sogeti is currently looking for a Azure Databricks Engineer to join our ACT Practice, ensuring alignment with business goals and driving impactful value creation for our clients.

Hybrid working: The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.

If you are successfully offered this position, you will go through a series of pre-employment checks, including identity, nationality (single or dual) or immigration status, employment history going back 3 continuous years, and unspent criminal record check (known as Disclosure and Barring Service)

Mandatory Skills

Azure Databricks Platform Engineering

• Hands‑on implementation and troubleshooting of Azure Databricks
• Databricks Serverless configuration and workload optimisation
• Databricks SQL and Delta Lake development
• Cluster management, compute policies, autoscaling, and workload monitoring

Data Engineering

• Strong Python, PySpark, and SQL development skills
• Design and build ingestion and transformation pipelines
• Full and incremental data loading
• CDC (Change Data Capture) implementation
• Data quality controls, schema evolution, reconciliation, and error handling

Integration Development

• REST API integration experience
• Integration with databases, SaaS applications, cloud storage, and enterprise systems
• Secure authentication and credential management

Azure Platform Services

• Azure Identity and Access Management
• Azure Networking and Private Endpoints
• Azure Security and Key Vault
• Azure Monitoring and Operational Support

FinOps & Cost Optimisation

• Resource tagging implementation
• Cost monitoring and reporting
• Budget controls and usage optimisation

Essential Skills

Platform Operations

• Experience supporting production Azure Databricks environments
• Monitoring, incident management, troubleshooting, and performance tuning
• Operational governance and platform standards implementation

Discovery Zone & Migration Support

• Experience enabling application deployment within Databricks
• BI tooling integration and scheduling
• Support for POSIT/RStudio migration activities

Collaboration

• Working with Data Engineers, Data Scientists, and Platform Teams
• Technical documentation and knowledge transfer
• Agile delivery experience

Desirable Skills

• POSIT/RStudio migration experience
• AI/ML and LLM integration
• RAG and Vector Database exposure
• Regulated industry experience

We Are a Disability Confident Employer

Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Government’s Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:

• Declare they have a disability, and
• Meet the minimum essential criteria for the role.

Please opt in during the application process.

What does 'Get The Future You Want' mean for you?

You will be empowered to explore, innovate, and progress. You will benefit from Capgemini’s ‘learning for life’ mindset, meaning you will have countless training and development opportunities from thinktanks to hackathons, and access to 250,000 courses with numerous external certifications from AWS, Microsoft, Harvard Management or, Cybersecurity qualifications and much more.

You will be encouraged to have a positive work-life balance. Our hybrid-first way of working means we embed hybrid working in all that we do and make flexible working arrangements the day‑to‑day reality for our people. All UK employees are eligible to request flexible working arrangements.

Why You Should Consider Capgemini

Growing clients’ businesses while building a more sustainable, more inclusive future is a tough ask. When you join Capgemini, you’ll join a thriving company and become part of a diverse collective of free‑thinkers, entrepreneurs and industry experts. We find new ways technology can help us reimagine what’s possible. It’s why, together, we seek out opportunities that will transform the world’s leading businesses, and it’s how you’ll gain the experiences and connections you need to shape your future. By learning from each other every day, sharing knowledge, and always pushing yourself to do better, you’ll build the skills you want. You’ll use your skills to help our clients leverage technology to innovate and grow their business. So, it might not always be easy, but making the world a better place rarely is.

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fuelled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion.