Azure Data Engineer (Fabric & Cloud Data Platform)

Webhosting; Greater London; Hybrid; GBP 85,000 - 120,000; Full time

Job summary

Rackspace Technology is seeking a highly skilled Azure Data Engineer to design, build, and maintain our enterprise data platform, implementing a modern data lakehouse on Fabric and Azure Databricks, with emphasis on security, logging, and access controls.

The role focuses on integrating existing pipelines, designing raw and curated zones, and ensuring robust encryption and governance to support analytics and data science, with on-site work in Central London.

Qualifications

• Experience building data lakehouse architectures on Azure Fabric and Databricks.
• Strong SQL and PySpark development skills.
• Experience with encryption, auditing, and access controls.

Responsibilities

• Design and optimize lakehouse architecture using Fabric OneLake and ADLS Gen2.
• Develop and tune large-scale batch pipelines with Azure Databricks.
• Orchestrate data workflows with Fabric Data Factory and Azure Data Factory.
• Ensure data security, logging, auditing, and RBAC/ABAC controls.
• Migrate legacy data pipelines into modern cloud formats.

Skills

Azure Data Engineering

Data Lakehouse Architecture

SQL

Python / PySpark

Security & Compliance

Tools

Azure Databricks

Fabric OneLake

Data Factory

ADLS Gen2

Microsoft Purview

Azure Key Vault

Job description

Job Description

Rackspace Technology™ is a leading end-to-end, hybrid cloud and AI solutions company. We can design, build, and operate our customers’ cloud environments across all major technology platforms, irrespective of technology stack or deployment model. We partner with our customers at every stage of their cloud journey, enabling them to modernize applications, build new products, and adopt innovative technologies. Named a best place to work by Newsweek and Forbes, we attract and develop world-class talent to deliver a Fanatical Experience™ – so our customers can achieve better outcomes and stay ahead of what’s next.

Job Summary

We are seeking a highly skilled Azure Data Engineer to design, build, and maintain our enterprise data platform. In this role, you will implement a modern data lakehouse architecture capable of supporting raw and curated data zones, scalable batch processing, and future analytical workloads. You will play a key role in integrating our existing data pipelines, models, and transformation logic into a unified ecosystem built on Microsoft Fabric and Azure Databricks. A heavy emphasis is placed on enforcing strict encryption, comprehensive logging, auditing, and fine-grained access controls to secure identifiable data.

Location: on-site with a customer in Central London.

Key Responsibilities

Data Lakehouse Architecture: Design and optimize a high-performance lakehouse pattern utilizing Microsoft Fabric OneLake and Azure Data Lake Storage Gen2 (ADLS Gen2).

Data Zone Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data.

Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark.

Orchestration & Workflow: Build automated data workflows using Microsoft Fabric Data Factory and Azure Data Factory to move data efficiently across the platform.

Legacy & Model Integration: Ensure seamless integration with existing data pipelines, legacy data models, and established transformation logic without breaking downstream analytical operations.

Platform Security & Governance: Implement enterprise-grade security baselines including data-at-rest/in-transit encryption, structured logging, automated audit trails, and strict access controls (RBAC, ABAC, and row/column-level security).

Future-Proof Analytics: Structure data foundations to readily support future analytical workloads, advanced reporting, and data science use cases.

Required Technical Skills

Cloud Platform & Fabric: 3+ years of dedicated hands-on experience in Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse).

Big Data Processing: Deep technical expertise with Apache Spark / Azure Databricks for distributed compute, memory optimization, and Delta Lake formats.

Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark).

Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview.

Data Integration: Proven track record of migrating, refactoring, or connecting legacy SQL/stored procedure logic and existing pipelines into modern cloud formats.