Data Engineer: SQL/Python on Databricks Lakehouse (Hybrid)

LA International; Swansea; Hybrid; GBP 74,000 - 129,000; Full time

Job summary

LA International is recruiting a Data Engineer in Swansea for a hybrid role with on-site 2 days per week. You will migrate legacy IMPALA and Oracle materialised views to Databricks Lakehouse Declarative Pipelines, translating SQL logic and building reconciliation checks. Experience with SQL is essential; some Python on Databricks (Azure) is a plus.

Source data in Unity Catalog; no data ingestion work. Security clearance is required; applicants with prior high-level clearance are welcome;

Qualifications

• Experience migrating SQL logic to lakehouse architectures.
• Familiarity with Unity Catalog and Databricks.
• Ability to build data quality and reconciliation checks.
• Azure Databricks experience preferred.

Responsibilities

• Migrate legacy IMPALA reporting to Databricks Lakehouse pipelines.
• Translate existing SQL logic into Lakehouse architecture.
• Build reconciliation checks to validate output against legacy system.
• Support pipelines post go-live and handle issues as they arise.

Skills

SQL

Python

Databricks

Azure

Unity Catalog

Tools

Databricks

Unity Catalog

Oracle

Job description

LA International is recruiting a Data Engineer in Swansea for a hybrid role with on-site 2 days per week. You will migrate legacy IMPALA and Oracle materialised views to Databricks Lakehouse Declarative Pipelines, translating SQL logic and building reconciliation checks. Experience with SQL is essential; some Python on Databricks (Azure) is a plus.

Source data in Unity Catalog; no data ingestion work. Security clearance is required; applicants with prior high-level clearance are welcome;