Senior Data QA Engineer: eFront to Databricks Migration

Morgan McKinley; City Of London; Presencial; GBP 65.000 - 110.000; Jornada completa

Descripción de la vacante

Morgan McKinley seeks a hands-on Senior Data QA / Data Quality Engineer to lead automated migration testing for an investment firm's eFront to Azure Databricks Lakehouse migration. You will own source-to-target reconciliation, validate financial calculations, and ensure 100% data integrity across Bronze/Silver/Gold layers.

Responsibilities include building automated frameworks with PyTest, PySpark, and advanced SQL, integrating checks into Azure DevOps pipelines, and monitoring schema drift and

Formación

• Hands-on experience testing data from complex pipelines and migrations.
• Experience with automated validation and reconciliation between source and target systems.
• Familiarity with private markets terminology and financial data structures.

Responsabilidades

• Lead automated migration testing and validation for data from eFront to Databricks Lakehouse.
• Design and implement source-to-target reconciliation queries and checks.
• Develop automated data quality tests using PyTest, PySpark, and advanced SQL.

Conocimientos

Python

PyTest

PySpark

Advanced SQL

Azure DevOps

Data validation

Reconciliation

Herramientas

Azure Databricks

Delta Lake

Descripción del empleo

Morgan McKinley seeks a hands-on Senior Data QA / Data Quality Engineer to lead automated migration testing for an investment firm's eFront to Azure Databricks Lakehouse migration. You will own source-to-target reconciliation, validate financial calculations, and ensure 100% data integrity across Bronze/Silver/Gold layers.

Responsibilities include building automated frameworks with PyTest, PySpark, and advanced SQL, integrating checks into Azure DevOps pipelines, and monitoring schema drift and