Senior Data Platform Engineer – Security Analytics (Hybrid)

Tesco Technology; Welwyn Garden City; Hybrid; GBP 85,000 - 120,000; Full time

Benefits offered by this job

Annual bonus up to 20%

Private medical insurance

26 weeks maternity/adoption leave with

24/7 virtual GP & EAP

Job summary

Tesco is seeking a Senior Data Platform Engineer to lead the design, build and maintenance of a robust data platform for advanced analytics, ML and GenAI in cybersecurity contexts.

You will architect real-time and batch data pipelines, integrate data from diverse security sources, and mentor junior engineers while collaborating across departments to deliver secure, scalable data services.

Qualifications

• Solid programming experience in Python and SQL.
• Experience building data platforms on cloud services (Databricks on Azure or GCP BigQuery).
• Knowledge of big data technologies (Spark, Flink).
• Solid grasp of Kubernetes, CI/CD and Terraform.
• Experience with ETL/ELT and orchestration tools like Airflow and dbt.
• Ability to guide and empower junior team members.
• Familiarity with machine learning and AI tools is desirable.

Responsibilities

• Design, build and manage realtime, near realtime and batch data architectures for threat detection and reporting.
• Implement automated data integration pipelines from various security sources into the data lake.
• Contribute to high-quality code through reviews and mentorship.
• Implement automation and DevOps practices for deployment and management of data platform components.
• Collaborate with cross-functional teams and communicate complex concepts clearly.

Skills

Python

SQL

Databricks

Spark

Flink

Kubernetes

Terraform

Airflow

dbt

CI/CD

Tools

Azure Databricks

Azure

GCP BigQuery

Spark

Flink

Kubernetes

Airflow

dbt

Terraform

CI/CD

Job description

Tesco is seeking a Senior Data Platform Engineer to lead the design, build and maintenance of a robust data platform for advanced analytics, ML and GenAI in cybersecurity contexts.

You will architect real-time and batch data pipelines, integrate data from diverse security sources, and mentor junior engineers while collaborating across departments to deliver secure, scalable data services.