Azure Databricks Engineer

Capgemini; United Kingdom; Hybrid; GBP 60,000 - 95,000; Full time

Benefits offered by this job

Hybrid working

Job summary

Capgemini is seeking an Azure Databricks Engineer to join our ACT Practice across the UK. The role focuses on hands-on Databricks engineering, data pipelines, and platform operations to deliver scalable data solutions for clients.

You will work closely with data engineers, scientists, and platform teams in a hybrid setup that blends Capgemini offices, client sites, and remote work. Ideal candidates will have strong Python/PySpark skills, Databricks expertise, and experience with SQL, Delta Lake,

Qualifications

• > Hands-on implementation and troubleshooting of Azure Databricks
• > Databricks Serverless configuration and workload optimisation
• > Databricks SQL and Delta Lake development
• > Cluster management, autoscaling, and workload monitoring
• > Strong Python, PySpark, and SQL development skills
• > Design and build ingestion and transformation pipelines
• > CDC implementation and data quality controls
• > Secure authentication and credential management, REST API integration
• > Azure Identity and Access Management, Private Endpoints, Key Vault
• > FinOps & cost monitoring and budgeting controls

Responsibilities

• Support production Azure Databricks environments with monitoring and incident management
• Collaborate with Data Engineers, Data Scientists, and Platform Teams
• Implement and maintain data ingestion and transformation pipelines
• Ensure data quality, schema evolution, and error handling
• Integrate BI tools and schedule data jobs
• Participate in cost optimisation and governance practices

Skills

Azure Databricks

Python

PySpark

SQL

REST API

Azure IAM

Azure Networking

Azure Key Vault

Cost Optimisation

Monitoring

Education

Bachelor’s degree or equivalent in a relevant field

Tools

Databricks

Job description

# Azure Databricks EngineerLondon, ManchesterApply for this job* Permanent* Experienced Professionals* Architecture* ID 555203-en\\_GB## **Internal Role Information**Grade – C1 Location -Anywhere in the UK## **About the job you're considering**We work closely with clients and partners to take full advantage of the opportunities of technology. We mobilize expert teams that create custom solutions from existing and emerging technology to deliver viable outcomes at speed. We call this 'Value in the making.'Part of the Capgemini Group, Sogeti makes business value through technology for organizations that need to implement innovation at speed and want a local partner with global scale. With a hands-on culture and close proximity to its clients, Sogeti is currently looking for a Azure Databricks Engineer to join our ACT Practice, ensuring alignment with business goals and driving impactful value creation for our clients.Hybrid working: The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.If you are successfully offered this position, you will go through a series of pre-employment checks, including identity, nationality (single or dual) or immigration status, employment history going back 3 continuous years, and unspent criminal record check (known as Disclosure and Barring Service)## **Mandatory Skills****Azure Databricks Platform Engineering*** Hands-on implementation and troubleshooting of Azure Databricks* Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion and transformation pipelines* Full and incremental data loading* CDC (Change Data Capture) implementation* Data quality controls, schema evolution, reconciliation, and error handling**Integration Development*** REST API integration experience* Integration with databases, SaaS applications, cloud storage, and enterprise systems* Secure authentication and credential management**Azure Platform Services*** Azure Identity and Access Management* Azure Networking and Private Endpoints* Azure Security and Key Vault* Azure Monitoring and Operational Support**FinOps & Cost Optimisation*** Resource tagging implementation* Cost monitoring and reporting* Budget controls and usage optimisation## **Essential Skills****Platform Operations*** Experience supporting production Azure Databricks environments* Monitoring, incident management, troubleshooting, and performance tuning* Operational governance and platform standards implementation**Discovery Zone & Migration Support*** Experience enabling application deployment within Databricks* BI tooling integration and scheduling* Support for POSIT/RStudio migration activities**Collaboration*** Working with Data Engineers, Data Scientists, and Platform Teams* Technical documentation and knowledge transfer* Agile delivery experience**Desirable Skills*** POSIT/RStudio migration experience* AI/ML and LLM integration* RAG and Vector Database exposure* Regulated industry experience## **We are a Disability Confident Employer:**Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Government’s Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who: • Declare they have a disability, and • Meet the minimum essential criteria for the role. Please opt in during the application process.## **What does 'Get The Future You Want' mean for you?**You will be empowered to explore, innovate, and progress. You will benefit from Capgemini’s ‘learning for life’ mindset, meaning you will have countless training and development opportunities from thinktanks to hackathons, and access to 250,000 courses with numerous external certifications from AWS, Microsoft, Harvard Management or, Cybersecurity qualifications and much more.You will be encouraged to have a positive work-life balance. Our hybrid-first way of working means we embed hybrid working in all that we do and make flexible working arrangements the day-to-day reality for our people. All UK employees are eligible to request flexible working arrangements.## **Why you should consider Capgemini:**Growing clients’ businesses while building a more sustainable, more inclusive future is a tough ask. When you join Capgemini, you’ll join a thriving company and become part of a diverse collective of free-thinkers, entrepreneurs and industry experts. We find new ways technology can help us reimagine what’s possible. It’s why, together, we seek out opportunities that will transform the world’s leading businesses, and it’s how you’ll gain the experiences and connections you need to shape your future. By learning from each other every day, sharing knowledge, and always pushing yourself to do better, you’ll build the skills you want. You’ll use your skills to help our clients leverage technology to innovate and grow their business. So, it might not always be easy, but making the world a better place rarely is.## **About Capgemini:**Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fuelled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion.