Lead Data Engineer: Large-Scale Pipelines & Production Readiness

GlobalLogic Inc.; Greater London; Hybrid; GBP 95,000 - 140,000; Full time

Benefits offered by this job

Hybrid work model

Private healthcare

Gym memberships

In-office yoga

Pension scheme

Volunteer day

Social events

Job summary

GlobalLogic in London is seeking a Principal Data Engineer to own the customer decisioning pipeline architecture, end-to-end orchestration, and production readiness. You will lead real-world targeting and eligibility rules, ensure data quality, and oversee downstream activation.

You’ll design scalable data pipelines, enforce testing standards, manage CI/CD with Git and Azure DevOps, and collaborate with CRM and marketing platforms.

Qualifications

• Deep expertise in building large-scale data engineering solutions with SQL, Python, Spark, and Databricks.
• Experience designing resilient workflows with dependency management and idempotent processing.
• Familiarity with CI/CD practices, code reviews, and releases using Git and Azure DevOps.

Responsibilities

• Design, build, and optimize scalable data pipelines that run on reliable scheduled and event-triggered workflows.
• Implement and review complex business logic for customer eligibility, contact policies, suppression rules, and cooldown periods.
• Oversee downstream activation datasets, file manifests, delivery status tracking, and secure SFTP platform integrations.
• Drive engineering standards by reviewing pull requests and enforcing rigorous unit, integration, and regression testing across environments.
• Lead production cutovers, automated alerting, disaster recovery, data reconciliation, and incident response.

Skills

Python

SQL

Apache Spark

Databricks

Tools

Git

Azure DevOps

Job description

GlobalLogic in London is seeking a Principal Data Engineer to own the customer decisioning pipeline architecture, end-to-end orchestration, and production readiness. You will lead real-world targeting and eligibility rules, ensure data quality, and oversee downstream activation.

You’ll design scalable data pipelines, enforce testing standards, manage CI/CD with Git and Azure DevOps, and collaborate with CRM and marketing platforms.