Data Engineer

Airswift; United Kingdom; Remote; GBP 60,000 - 90,000; Full time

Job summary

Airswift is seeking a Data Engineer for a 12-month contract in the United Kingdom with remote work options. The role focuses on building and maintaining large-scale data pipelines using Databricks, PySpark and SQL, handling transaction and loyalty data, and ensuring robust production stability.

You will contribute to data integration, migration stabilisation, orchestration, monitoring and data quality across critical datasets, delivering reliable cloud-based solutions.

Qualifications

• Experience delivering and maintaining large-scale data pipelines in production environments.
• Experience with transaction, payment, loyalty, EPOS/WRE and identity-matching data.
• Experience with historical, full and incremental loads, high-water marks and reconciliation processes.
• Experience in production troubleshooting, dependency analysis, pipeline repointing and migration stabilisation.
• Experience supporting workflow orchestration, retry and restart patterns, and operational handover into business-as-usual support.
• Experience with integration testing, production validation and data completeness investigations.

Responsibilities

• Develop and maintain production-grade data pipelines using SQL, Python, PySpark and Databricks.
• Support data migration, orchestration, monitoring and data quality across datasets.
• Troubleshoot production issues, perform dependency analysis and manage pipeline reruns and migrations.
• Contribute to testing, validation and handover to BAU teams.

Skills

SQL

Python

PySpark

Databricks

Git

CI/CD

Tools

Git

CI/CD

SFTP

Job description

Job Title: Data Engineer Duration: 12 months Schedule: 40 hrs/week Location: United Kingdom (Remote) This opportunity offers the chance to work on complex data engineering initiatives focused on Databricks, PySpark, SQL and large-scale transaction and loyalty data pipelines.

The role will support production-ready integration, migration stabilisation, orchestration, monitoring and data quality across critical datasets and workflows. This position is well suited to professionals who enjoy solving pipeline reliability challenges, improving reconciliation processes and delivering robust cloud-based data solutions.

Required Experience

• Experience delivering and maintaining large-scale data pipelines in production environments
• Experience working with transaction, payment, loyalty, EPOS/WRE and identity-matching data
• Experience with historical, full and incremental loads, high-water marks and reconciliation processes
• Experience in production troubleshooting, dependency analysis, pipeline repointing and migration stabilisation
• Experience supporting workflow orchestration, retry and restart patterns, and operational handover into business-as-usual support
• Experience with integration testing, production validation and data completeness investigations

Required Skills

• Strong SQL, Python, PySpark and Databricks pipeline engineering skills
• Hands-on ETL/ELT development across cloud-based data platforms
• Workflow orchestration and dependency management
• Idempotent processing, failure recovery, error handling and controlled reloads
• Monitoring, logging, alerting, data-quality checks and freshness controls
• Git, pull requests, unit testing, CI/CD and release support
• API, file-based and SFTP integration patterns
• Technical design preparation and structured testing support

Preferred Qualifications

• Exposure to reporting or Power BI datasets
• Delta Lake or Delta MERGE knowledge
• Databricks Asset Bundles experience
• Azure DevOps experience
• MLOps or model deployment experience
• Knowledge of SFMC or Comarch integration
• Experience with customer decisioning or personalisation platforms
• Experience with AWS-to-Databricks migration
• Exposure to Informatica, SSIS or legacy ETL environments
• Multi-market SCI knowledge

About Airswift

Airswift is an international workforce solutions provider within the energy, process and infrastructure industries. Airswift serves as a strategic partner to clients, offering a turnkey workforce solution to capture and deliver the top talent needed to complete successful projects by aligning with unique project needs. With over 1000 employees and 9,000 contractors operating in over 60 countries, Airswift's geographical reach and pool of talent available is unmatched in the industry.