Azure Databricks Engineer

Capgemini; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

Capgemini in the UK is seeking an Azure Databricks Engineer to join our ACT Practice, delivering data engineering solutions aligned with client goals. The role involves hands-on implementation, troubleshooting, and integration across Azure platform services.

You will work in a hybrid setup with a mix of client sites, Capgemini offices and home-based work, collaborating with data teams to drive value and innovation for diverse clients.

Qualifications

• Experience supporting production Azure Databricks environments.
• Design and build ingestion and transformation pipelines.
• Strong Python, PySpark, and SQL development skills.
• Data engineering with Delta Lake and CDC implementations.

Responsibilities

• Support production Azure Databricks environments with monitoring and incident response.
• Design and implement data ingestion and transformation pipelines.
• Collaborate with Data Engineers and Data Scientists to deliver robust data solutions.

Skills

Platform Operations

Agile delivery

Collaboration

Tools

Azure Databricks

Python

PySpark

SQL

Job description

Internal Role InformationGrade – C1Location -Anywhere in the UKAbout the job you're consideringWe work closely with clients and partners to take full advantage of the opportunities of technology. We mobilize expert teams that create custom solutions from existing and emerging technology to deliver viable outcomes at speed. We call this 'Value in the making.'Part of the Capgemini Group, Sogeti makes business value through technology for organizations that need to implement innovation at speed and want a local partner with global scale. With a hands-on culture and close proximity to its clients, Sogeti is currently looking for a Azure Databricks Engineer to join our ACT Practice, ensuring alignment with business goals and driving impactful value creation for our clients.Hybrid working: The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.If you are successfully offered this position, you will go through a series of pre-employment checks, including identity, nationality (single or dual) or immigration status, employment history going back 3 continuous years, and unspent criminal record check (known as Disclosure and Barring Service)Mandatory SkillsAzure Databricks Platform EngineeringHands-on implementation and troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental data loadingCDC (Change Data Capture) implementationData quality controls, schema evolution, reconciliation, and error handlingIntegration DevelopmentREST API integration experienceIntegration with databases, SaaS applications, cloud storage, and enterprise systemsSecure authentication and credential managementAzure Platform ServicesAzure Identity and Access ManagementAzure Networking and Private EndpointsAzure Security and Key VaultAzure Monitoring and Operational SupportFinOps & Cost OptimisationResource tagging implementationCost monitoring and reportingBudget controls and usage optimisationEssential SkillsPlatform OperationsExperience supporting production Azure Databricks environmentsMonitoring, incident management, troubleshooting, and performance tuningOperational governance and platform standards implementationDiscovery Zone & Migration SupportExperience enabling application deployment within DatabricksBI tooling integration and schedulingSupport for POSIT/RStudio migration activitiesCollaborationWorking with Data Engineers, Data Scientists, and Platform TeamsTechnical documentation and knowledge transferAgile delivery experienceDesirable SkillsPOSIT/RStudio migration experienceAI/ML and LLM integrationRAG and Vector Database exposureRegulated industry experienceWe are a Disability Confident Employer:Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Government’s Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:• Declare they have a disability, and• Meet the minimum essential criteria for the role.Please opt in during the application process.What does 'Get The Future You Want' mean for you?You will be empowered to explore, innovate, and progress. You will benefit from Capgemini’s ‘learning for life’ mindset, meaning you will have countless training and development opportunities from thinktanks to hackathons, and access to 250,000 courses with numerous external certifications from AWS, Microsoft, Harvard Management or, Cybersecurity qualifications and much more.You will be encouraged to have a positive work-life balance. Our hybrid-first way of working means we embed hybrid working in all that we do and make flexible working arrangements the day-to-day reality for our people. All UK employees are eligible to request flexible working arrangements.Why you should consider Capgemini:Growing clients’ businesses while building a more sustainable, more inclusive future is a tough ask. When you join Capgemini, you’ll join a thriving company and become part of a diverse collective of free-thinkers, entrepreneurs and industry experts. We find new ways technology can help us reimagine what’s possible. It’s why, together, we seek out opportunities that will transform the world’s leading businesses, and it’s how you’ll gain the experiences and connections you need to shape your future. By learning from each other every day, sharing knowledge, and always pushing yourself to do better, you’ll build the skills you want. You’ll use your skills to help our clients leverage technology to innovate and grow their business. So, it might not always be easy, but making the world a better place rarely is.About Capgemini:Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fuelled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion.