Azure Data Lakehouse Engineer — Scalable, Secure Pipelines

Pace Industries, LLC; Greater London; Hybrid; GBP 61,000 - 81,000; Full time

Job summary

Rackspace Technology is seeking an Azure Data Engineer to design, build, and maintain our enterprise data platform. You will implement a modern lakehouse architecture on Microsoft Fabric and Azure Databricks, with strict encryption, logging, auditing, and fine-grained access controls for identifiable data.

Location: on-site with a customer in Central London. You will work on data lakehouse design, data zone management, scalable batch processing with Spark-based pipelines, and security governance

Qualifications

• 3+ years of dedicated hands-on experience in Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse).
• Deep technical expertise with Apache Spark / Azure Databricks for distributed compute, memory optimization, and Delta Lake formats.
• Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark).
• Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview.
• Proven track record of migrating, refactoring, or connecting legacy SQL/stored procedure logic and existing pipelines into modern cloud formats.

Responsibilities

• Data Lakehouse Architecture: design and optimize a high-performance lakehouse pattern utilizing Microsoft Fabric OneLake and Azure Data Lake Storage Gen2 (ADLS Gen2).
• Data Zone Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data.
• Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark.
• Orchestration & Workflow: Build automated data workflows using Microsoft Fabric Data Factory and Azure Data Factory to move data efficiently across the platform.
• Legacy & Model Integration: Ensure seamless integration with existing data pipelines, legacy data models, and established transformation logic without breaking downstream analytical operations.
• Platform Security & Governance: Implement enterprise-grade security baselines including data-at-rest/in-transit encryption, structured logging, automated audit trails, and strict access controls (RBAC, ABAC, and row/column-level security).
• Future-Proof Analytics: Structure data foundations to readily support future analytical workloads, advanced reporting, and data science use cases.

Skills

Azure

Fabric OneLake

Azure Data Factory

Azure Databricks

PySpark

SQL (T-SQL, SparkSQL)

Python

RBAC/ABAC

Tools

Terraform

Bicep

Git/Azure DevOps

Job description

Rackspace Technology is seeking an Azure Data Engineer to design, build, and maintain our enterprise data platform. You will implement a modern lakehouse architecture on Microsoft Fabric and Azure Databricks, with strict encryption, logging, auditing, and fine-grained access controls for identifiable data.

Location: on-site with a customer in Central London. You will work on data lakehouse design, data zone management, scalable batch processing with Spark-based pipelines, and security governance