Azure Databricks Data Engineer | Remote & Secure Cloud

Atos; Wideopen; On-site; GBP 70,000 - 110,000; Full time

Benefits offered by this job

25 days leave + holidays

Pension scheme

Private medical cover

Income protection

Life assurance

Flex benefits program

Job summary

Atos UK is seeking an Azure Databricks Developer to design, develop, and support scalable data engineering solutions using Azure Databricks and the broader Azure data platform. In this role, you will build robust data ingestion, transformation, and processing pipelines that enable advanced analytics, artificial intelligence, machine learning, and business intelligence capabilities.

Working closely with Data Architects, Data Engineers, Data Analysts, and Solution Architects, you will help deliver

Qualifications

• Expertise with Azure Databricks, Apache Spark, PySpark, Spark SQL.
• Strong Python and SQL development skills.
• Experience building enterprise-scale ETL/ELT pipelines within Azure environments.

Responsibilities

• Design, develop, and optimise scalable data pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake.
• Build robust ETL/ELT solutions and data ingestion frameworks supporting batch and streaming workloads.
• Integrate data from multiple sources including Azure SQL Database, Azure Data Lake Storage Gen2, Microsoft Fabric, APIs, SaaS platforms, and on-premise systems.
• Implement data quality, validation, monitoring, and governance controls across the data lifecycle.
• Support platform engineering activities including CI/CD, environment promotion, performance optimization, and operational support.

Skills

Azure Databricks

Apache Spark

PySpark

Spark SQL

Python

SQL

Azure Data Lake Storage Gen2

Azure Data Factory

Job description

Atos UK is seeking an Azure Databricks Developer to design, develop, and support scalable data engineering solutions using Azure Databricks and the broader Azure data platform. In this role, you will build robust data ingestion, transformation, and processing pipelines that enable advanced analytics, artificial intelligence, machine learning, and business intelligence capabilities.

Working closely with Data Architects, Data Engineers, Data Analysts, and Solution Architects, you will help deliver