Senior Data Platform Engineer: Cyber Analytics & GenAI

Tesco UK; Digswell; Hybrid; GBP 90,000 - 120,000; Full time

Job summary

Qualifications

• Solid programming experience with Python and SQL.
• Experience building data platforms on cloud services like Databricks on Azure or GCP BigQuery and big data technologies like Spark, Flink.
• Solid grasp of Kubernetes, CI/CD and Terraform
• Working knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbt
• Ability to provide input, guide, empower junior members of the team to achieve desired outcomes
• Familiarity with machine learning and AI tools is desirable
• Knowledge of cybersecurity principles and practices

Responsibilities

• Design, build and manage realtime, near realtime and batch data architectures that support threat detection, incident response and reporting through advanced analytics, machine learning and GenAI capabilities.
• Implement and manage the automated frameworks for integrating data from various security sources into the security data lake. Also, the data transformation frameworks to move data between the raw, trusted, and curated data layers.
• Contribute to raising the quality bar of the team's codebase by producing high-quality code, conducting thorough peer reviews, and proactively providing constructive feedback to other team members on their code.
• Implement various automations and DevOps practices to streamline the deployment, configuration, and management of data platform components.
• Collaborate with teams from various departments, effectively communicating technical concepts and solutions to individuals with different levels of technical expertise.

Skills

Python

SQL

Databricks Azure

BigQuery

Spark

Flink

Kubernetes

CI/CD

Terraform

Airflow

dbt

Machine learning

Cybersecurity

Tools

Terraform

Airflow

Job description