Senior Data Platform Engineer

Tesco Technology; Welwyn Garden City; Hybrid; GBP 85,000 - 120,000; Full time

Benefits offered by this job

Annual bonus up to 20%

Private medical insurance

26 weeks maternity/adoption leave with

24/7 virtual GP & EAP

Job summary

Tesco is seeking a Senior Data Platform Engineer to lead the design, build and maintenance of a robust data platform for advanced analytics, ML and GenAI in cybersecurity contexts.

You will architect real-time and batch data pipelines, integrate data from diverse security sources, and mentor junior engineers while collaborating across departments to deliver secure, scalable data services.

Qualifications

• Solid programming experience in Python and SQL.
• Experience building data platforms on cloud services (Databricks on Azure or GCP BigQuery).
• Knowledge of big data technologies (Spark, Flink).
• Solid grasp of Kubernetes, CI/CD and Terraform.
• Experience with ETL/ELT and orchestration tools like Airflow and dbt.
• Ability to guide and empower junior team members.
• Familiarity with machine learning and AI tools is desirable.

Responsibilities

• Design, build and manage realtime, near realtime and batch data architectures for threat detection and reporting.
• Implement automated data integration pipelines from various security sources into the data lake.
• Contribute to high-quality code through reviews and mentorship.
• Implement automation and DevOps practices for deployment and management of data platform components.
• Collaborate with cross-functional teams and communicate complex concepts clearly.

Skills

Python

SQL

Databricks

Spark

Flink

Kubernetes

Terraform

Airflow

dbt

CI/CD

Tools

Azure Databricks

Azure

GCP BigQuery

Spark

Flink

Kubernetes

Airflow

dbt

Terraform

CI/CD

Job description

About the role

As the Senior Data Platform Engineer in the Cyber Data Platform team, you will play a key role in building and maintaining a robust data platform that enables users to develop advanced analytics, machine learning, and GenAI solutions to strengthen our security defences. You will collaborate closely with cross functional data teams to ensure the platform is scalable, secure, and aligned with cybersecurity goals. Also, you will contribute to the data platform architecture and mentor junior team members to deliver high-quality data services.

You will be responsible for

Data Architecture: Design, build and management of realtime, near realtime and batch data architectures that support threat detection, incident response and reporting through advanced analytics, machine learning and GenAI capabilities.

Data Integration and Transformation: Implement and manage the automated frameworks for integrating data from various security sources into the security data lake. Also, the data transformation frameworks to move data between the raw, trusted, and curated data layers.

Coding: Contribute to raising the quality bar of the team's codebase by producing high-quality code, conducting thorough peer reviews, and proactively providing constructive feedback to other team members on their code.

Automation and DevOps: Implement various automations and DevOps practices to streamline the deployment, configuration, and management of data platform components.

Collaboration and Communication: Collaborate with teams from various departments, effectively communicating technical concepts and solutions to individuals with different levels of technical expertise.

Knowledge of cybersecurity principles and practices.

You will need

Solid programming experience with programming languages like Python along with proficiency in SQL.

Experience building data platforms on cloud services like Databricks on Azure or GCP BigQuery and big data technologies like Spark, Flink etc.

Solid grasp of Kubernetes, CI/CD and Terraform.

Working knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbt.

Ability to provide clear input, guide, empower junior members of the team to achieve desired outcomes.

Familiarity with machine learning and AI tools is desirable.

Whats in it for you?

We’re all about the little helps. That’s why we make sure our Tesco colleague benefits package takes care of you – both in and out of work.

• Annual bonus scheme of up to 20% of base salary.
• Holiday starting at 25 days plus a personal day (plus Bank holidays).
• Private medical insurance.
• 26 weeks maternity and adoption leave (12 months service required at the qualifying date) at full pay, followed by 13 weeks of Statutory Maternity Pay or Statutory Adoption Pay, we also offer 6 weeks fully paid paternity leave.
• Free 24/7 virtual GP service, Employee Assistance Programme (EAP) for you and your family, free access to a range of experts to support your mental wellbeing.
• This information is a shortened summary, refer to our policies for full details.

About Us

You might know us as a supermarket, technology company or even for our award-winning mobile network. Truth is, we’re all of those things, and much more. Our colleagues work with one goal in mind, helping to make every day a little better for our customers, colleagues and communities all over the world. No two customers are the same, neither are our colleagues.

At Tesco, we champion a balance that lets you thrive both in and out of work. Spend 60% of your week collaborating with colleagues at our office locations or local sites and the rest remotely. Whether you're just kicking off your career, juggling passions, or navigating big life events, we’re here to support you. We always welcome a conversation about flexible working, so talk to us throughout your application about how we can support.

We're proud to be an accredited Disability Confident Leader, where everyone’s welcome. That’s why we commit to providing a fully inclusive and accessible recruitment process. And if you're interested in joining our team but don't tick every box, don't let that hold you back from applying.