Data Engineer

Airswift; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

Airswift is seeking a highly skilled Data Engineer to design and deliver robust, high-performance data pipelines for business-critical insights in London. The role focuses on building cloud-based data platforms and analytics capabilities using Azure Databricks and Data Factory.

You'll collaborate with stakeholders, drive scalable ETL/ELT processes, and work within Agile delivery teams to ensure quality, maintainability, and timely delivery over an initial 12-month contract (Inside IR35) with

Qualifications

• 6+ years of experience in Data Engineering or related fields.
• Experience with Azure Databricks and Azure Data Factory.
• Strong ETL/ELT design and data pipeline development.
• Proficiency in Scala and distributed data processing.
• Experience in Azure cloud environments and building scalable data platforms.
• Good communication and stakeholder management in Agile teams.

Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Azure Databricks.
• Build and optimize enterprise-grade data pipelines with Azure Data Factory.
• Process and transform large-scale datasets containing millions of records.
• Drive performance tuning in distributed data processing environments.
• Collaborate with stakeholders to translate business needs into technical solutions.
• Participate in architecture discussions and contribute to best practices.

Skills

Azure Databricks

Azure Data Factory

Scala

ETL/ELT design

Data pipelines

Azure cloud

Distributed processing

Agile delivery

Job description

Duration: 12 months (with potential extension)

Workload: Full-time hours

Setup: Freelance Inside ir35

Location: London Hybrid – (3 days onsite, 2 days remote)

Overview:

We are seeking a highly skilled Data Engineer to design and deliver robust, high-performance data pipelines and enable advanced analytics for business-critical insights.

The Role

As a Data Engineer, you will play a key role in designing, developing, and maintaining robust cloud-based data solutions. You'll collaborate closely with technical and business stakeholders to build efficient, reliable, and scalable data platforms that enable data-driven decision making.

You'll work with cutting-edge Azure technologies, leveraging Databricks and Data Factory to create high-performing data pipelines and processing frameworks.

Key Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Azure Databricks
• Build and optimize enterprise-grade data pipelines using Azure Data Factory
• Process and transform large-scale datasets containing millions of records
• Drive performance tuning and optimization across distributed data processing environments
• Collaborate with stakeholders to gather requirements and translate business needs into technical solutions
• Participate in architecture discussions and contribute to best practice implementation
• Ensure high standards of code quality, testing, documentation, and maintainability
• Identify and resolve technical issues, escalating risks where appropriate
• Work within Agile delivery teams to deliver high-quality data solutions

Required Experience

• 6+ years of experience in Data Engineering, Data Platform Engineering, or related fields
• Strong hands-on experience with Azure Databricks
• Deep understanding of Azure Data Factory
• Proven experience developing and optimizing large-scale ETL/ELT processes
• Strong knowledge of data processing concepts and distributed computing
• Hands-on experience with Scala
• Experience working within Microsoft Azure cloud environments
• Strong analytical and problem-solving skills
• Excellent communication and stakeholder management abilities