Platform Engineer

TalkTalk; England; On-site; GBP 45,000 - 65,000; Full time

Job summary

A leading telecommunications company in the UK is looking for a skilled Data Engineer. The role involves designing and maintaining Azure infrastructure using Terraform, developing CI/CD pipelines in Azure DevOps, and working with Databricks to support data processing. Candidates must have proven expertise in Microsoft Azure and deep experience with Databricks and Terraform. The position provides excellent growth opportunities within a supportive team environment.

Qualifications

• Proven expertise with Microsoft Azure networking, security, storage, and compute.
• Strong proficiency in Databricks with hands-on Scala (Spark) and PySpark.
• Deep experience with Terraform for Azure resource deployment.
• Hands-on with Azure DevOps pipelines (YAML, agents).
• Familiarity with monitoring and logging in Azure.

Responsibilities

• Design, build, and maintain Azure infrastructure with Terraform.
• Develop and optimise CI / CD pipelines in Azure DevOps.
• Work with Databricks for data ingestion and processing.
• Implement secure connectivity and governance best practices.
• Monitor and optimise platform reliability.

Skills

Microsoft Azure

Databricks (Scala, PySpark)

Terraform

Azure DevOps

Azure Active Directory / Entra ID

Problem-solving

Collaboration

Job description

12m FTC

The Team

The Data Engineering team is positioned in the Delivery business unit of the IT function within the Data & Automation department sitting alongside the Report Engineering, Data Governance, Data Migrations and Automation teams.

Data Engineering is a perfect blend of Data Engineers and QA Engineers that are renowned for their forward thinking and trail blazing. Exceptionally talented, supportive and personable, it’s an area that’s challenging, provides excellent growth opportunities and rewarding.

Role Expectations

• Design, build, and maintain Azure infrastructure with Terraform (IaC) as the single source of truth.
• Develop and optimise CI / CD pipelines in Azure DevOps to automate deployment of workspaces, Unity Catalog, networking, and security.
• Work with Databricks (Spark / Scala, PySpark) to support ingestion frameworks, data processing, and platform-level libraries.
• Implement secure connectivity (VNET injection, Private Link, firewall, DNS, RBAC).
• Monitor, troubleshoot, and optimise platform reliability and performance.
• Partner with architects, engineers, and product teams to enforce best practices in governance, security, and automation.
• Document infrastructure, processes, and standards for wider engineering adoption.

Must Have

• Proven expertise with Microsoft Azure (networking, security, storage, compute).
• Strong proficiency in Databricks with hands‑on Scala (Spark) and PySpark.
• Deep experience with Terraform for Azure resource deployment and governance.
• Hands‑on with Azure DevOps pipelines (YAML, agents, service connections).
• Understanding of Azure Active Directory / Entra ID, RBAC, and service principals.
• Familiarity with monitoring and logging in Azure (Log Analytics, Monitor, Application Insights).
• Excellent problem‑solving and collaboration skills.
• Experience in large‑scale enterprise environments and agile delivery.