SC Data Engineers (SQL, PYTHON, AZURE, SSIS, SSAS)

LA International; Swansea; On-site; GBP 74,000 - 129,000; Full time

Job summary

LA International is recruiting a Data Engineer in Swansea for a hybrid role with on-site 2 days per week. You will migrate legacy IMPALA and Oracle materialised views to Databricks Lakehouse Declarative Pipelines, translating SQL logic and building reconciliation checks. Experience with SQL is essential; some Python on Databricks (Azure) is a plus.

Source data in Unity Catalog; no data ingestion work. Security clearance is required; applicants with prior high-level clearance are welcome;

Qualifications

• Experience migrating SQL logic to lakehouse architectures.
• Familiarity with Unity Catalog and Databricks.
• Ability to build data quality and reconciliation checks.
• Azure Databricks experience preferred.

Responsibilities

• Migrate legacy IMPALA reporting to Databricks Lakehouse pipelines.
• Translate existing SQL logic into Lakehouse architecture.
• Build reconciliation checks to validate output against legacy system.
• Support pipelines post go-live and handle issues as they arise.

Skills

SQL

Python

Databricks

Azure

Unity Catalog

Tools

Databricks

Unity Catalog

Oracle

Job description

Our government client is looking for Data Engineers to begin ASAP. Please see below for role description and details.

• Hybrid, 2 days per week on site in Swansea, Wales
• Inside IR35
• SC clearance required
• 3-6 months initially + longer extensions expected

Role details:

To migrate legacy IMPALA reporting (Oracle materialised views and Discoverer workbooks) to Databricks Lakeflow Declarative Pipelines. The work involves translating existing SQL logic into a lakehouse architecture, building reconciliation checks to validate output against the legacy system, and supporting the pipelines once live.

It is mostly SQL knowledge but some familiarity with Python on Databricks (in Azure) will be beneficial. Source data is already available in Databricks - you would be working with tables in Unity Catalog, not managing data ingestion or connectivity.

Due to the nature and urgency of this post, candidates holding or who have held high level security clearance in the past are most welcome to apply. Please note successful applicants will be required to be security cleared prior to appointment which can take a minimum 10 weeks. LA International is an award-winning partner of choice for many of the world\'s most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance.

An award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over multiple years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience.

Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.