Consultant Data Engineer IRC302341

GlobalLogic; Greater London; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Health and wellness plan

Private healthcare

Gym membership discounts

In‑office yoga sessions

Pension scheme

Volunteer day

Social events

Job summary

GlobalLogic is seeking a Consultant Data Engineer to lead a major architectural shift from on‑prem Teradata to a cutting‑edge AWS Lakehouse. You will build robust data pipelines, modernize legacy code with dbt, and govern warehousing using Databricks.

You will collaborate with System Analysts, Automation Engineers, and Data Architects to convert hundreds of legacy scripts, orchestrate them with Astronomer, and ensure end‑to‑end data lineage for regulatory compliance.

Qualifications

• 7+ years of dedicated Data Engineering experience, with complex migrations from On‑Prem to Cloud.
• Experience in Financial Services environment, especially Risk/Finance domains.
• Strong CI/CD for data pipelines and Git version control; data modelling expertise.

Responsibilities

• Build and maintain scalable data pipelines for moving data to the lakehouse.
• Transform up to 500 legacy Teradata BTEQ scripts into modular dbt models.
• Design, develop and monitor DAGs with Astronomer/Airflow for end‑to‑end workflows.
• Integrate pipelines with OpenLineage for full data traceability and governance.
• Support QA/Automation teams to ensure data parity and accurate reconciliation.

Skills

SQL

Python

Airflow

AWS

Databricks

dbt

Teradata

CI/CD

Git

Terraform

OpenLineage

Data Modeling

Tools

Apache Airflow

Astronomer

Apache Iceberg

OpenLineage

Teradata

Job description

Description

We are seeking a highly skilled Сonsultant Data Engineer to be the driving force behind this massive architectural shift. You will be hands‑on in moving a highly complex, highly regulated data ecosystem from an on‑premise Teradata environment to a cutting‑edge AWS “Lakehouse” architecture.

Description

We are seeking a highly skilled Сonsultant Data Engineer to be the driving force behind this massive architectural shift. You will be hands‑on in moving a highly complex, highly regulated data ecosystem from an on‑premise Teradata environment to a cutting‑edge AWS “Lakehouse” architecture.

In this role, you will focus on building robust data pipelines, modernizing legacy code, and ensuring flawless data delivery. You will work closely with System Analysts, Automation Engineers, and Data Architects to convert hundreds of legacy scripts into modern dbt models, orchestrate them via Astronomer (Airflow), and establish governed warehousing in Databricks.

Requirements

• Cloud Platforms: Hands‑on engineering experience within the AWS ecosystem (S3, IAM, Glue) and deep expertise in Databricks (SparkSQL, PySpark, governed cataloging).
• Modern Data Stack: Strong proficiency in dbt (Core or Cloud) for data transformation and understanding of modern table formats, specifically Apache Iceberg.
• Orchestration: Experience building and maintaining data pipelines using Apache Airflow (Astronomer experience is a strong plus).
• Languages: Expert‑level SQL (ability to read Teradata SQL/BTEQ and write highly optimised Spark/Databricks SQL) and strong Python skills for pipeline development and Airflow DAGs.
• Legacy Systems: Familiarity with Teradata architecture and utilities (FastExport, BTEQ) to effectively understand the source systems being decommissioned.

Experience & Qualifications

• Experience: 7+ years of dedicated Data Engineering experience, with a proven track record in complex data migration projects (On‑Premise to Cloud).
• Financial Services Background: Experience working in a highly regulated banking or financial services environment is strongly preferred, particularly within Risk or Finance domains.
• Engineering Best Practices: Strong grasp of CI/CD for data pipelines, Git version control, data modelling (Kimball/Inmon), and infrastructure as code (Terraform).
• Communication: Ability to work collaboratively in a fast‑paced, cross‑functional agile team, effectively communicating technical blockers to technical leads and project managers.

Job responsibilities

• Lakehouse Engineering: Build and maintain scalable data pipelines to migrate Consumer Risk datasets from Teradata to Amazon S3, utilizing Apache Iceberg for the lakehouse layer and Databricks for governed warehousing.
• Code Modernisation (BTEQ to dbt): Perform complex code conversions, transforming up to 500 legacy Teradata BTEQ scripts into modular, version‑controlled, and highly performant dbt (data build tool) models.
• Pipeline Orchestration: Design, develop, and monitor complex Directed Acyclic Graphs (DAGs) using Astronomer (Apache Airflow) to orchestrate end‑to‑end data workflows.
• Data Lineage & Transparency: Integrate pipelines with OpenLineage to ensure complete, end‑to‑end data traceability, providing the transparency required for regulatory compliance.
• Reconciliation & Parity: Work alongside the QA/Automation team to execute automated reconciliation frameworks, debugging and resolving data discrepancies to ensure strict attribute‑level parity and timing equivalence during governed parallel runs.

What we offer

Our goal is to build an inclusive positive culture where everyone can feel comfortable being themselves, empowering people to create their own high standards and therefore more value. We work together to promote fairness while recognising, valuing and embracing differences – providing a transparent support structure and generous training budget to help our people develop skills to progress their career. Our region supports a hybrid model that can flex across a wide spectrum of working options determined by business, customer and individual needs.

You’ll benefit from a comprehensive health and wellness plan, private healthcare (clinical and mental wellbeing), discounted gym memberships, and in‑office yoga sessions and massages. We offer a fantastic benefits package including a competitive pension scheme and recognition schemes through bonus/reward initiatives. Colleagues are entitled to an annual volunteering day – so you can take time to support a cause close to your heart. We also love to stay social with trips to the zoo, quiz nights, sports events, theatre trips and much more.

We are an equal opportunities employer. It is our policy to promote an environment free from discrimination, harassment and victimisation, and we actively welcome applications from people with a disability or long‑term health condition. As users of the Disability Confident scheme, we guarantee to interview all disabled applicants who meet the minimum essential criteria for this vacancy.

We are committed to providing reasonable adjustments. If you require any adjustments for the application process, the interview, or the job itself, or if you need this information in an accessible format, please contact uki‑[e-mail].

About GlobalLogic

GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world’s largest and most forward‑thinking companies. Since 2000, we’ve been at the forefront of the digital revolution – helping create some of the most innovative and widely used digital products and experiences. Today we continue to collaborate with clients in transforming businesses and redefining industries through intelligent products, platforms, and services.