Remote Data Engineer - Databricks & PySpark Pipelines

Airswift; United Kingdom; Remote; GBP 60,000 - 90,000; Full time

Job summary

Airswift is seeking a Data Engineer for a 12-month contract in the United Kingdom with remote work options. The role focuses on building and maintaining large-scale data pipelines using Databricks, PySpark and SQL, handling transaction and loyalty data, and ensuring robust production stability.

You will contribute to data integration, migration stabilisation, orchestration, monitoring and data quality across critical datasets, delivering reliable cloud-based solutions.

Qualifications

• Experience delivering and maintaining large-scale data pipelines in production environments.
• Experience with transaction, payment, loyalty, EPOS/WRE and identity-matching data.
• Experience with historical, full and incremental loads, high-water marks and reconciliation processes.
• Experience in production troubleshooting, dependency analysis, pipeline repointing and migration stabilisation.
• Experience supporting workflow orchestration, retry and restart patterns, and operational handover into business-as-usual support.
• Experience with integration testing, production validation and data completeness investigations.

Responsibilities

• Develop and maintain production-grade data pipelines using SQL, Python, PySpark and Databricks.
• Support data migration, orchestration, monitoring and data quality across datasets.
• Troubleshoot production issues, perform dependency analysis and manage pipeline reruns and migrations.
• Contribute to testing, validation and handover to BAU teams.

Skills

SQL

Python

PySpark

Databricks

Git

CI/CD

Tools

Git

CI/CD

SFTP

Job description

Airswift is seeking a Data Engineer for a 12-month contract in the United Kingdom with remote work options. The role focuses on building and maintaining large-scale data pipelines using Databricks, PySpark and SQL, handling transaction and loyalty data, and ensuring robust production stability.

You will contribute to data integration, migration stabilisation, orchestration, monitoring and data quality across critical datasets, delivering reliable cloud-based solutions.