Azure Data Engineer — Scalable Data Pipelines for AI

Collective Work; Greater London; On-site; GBP 92,000 - 166,000; Part time

Job summary

Collective.work is building the next generation of AI-powered recruitment. We are seeking a freelance Data Engineer to design, build, and maintain scalable data pipelines in the Microsoft Azure ecosystem from London.

You will work with Azure Data Factory, Databricks, PySpark, SQL, and Python to ingest, transform, and enrich data for AI agents and product features. This contract role starts ASAP, with the day rate to be agreed.

Qualifications

• Experience designing scalable data pipelines.
• Proficiency with Azure data services.
• Strong SQL, Python, PySpark.
• Ability to improve data quality and performance.

Responsibilities

• Design, build, and maintain scalable data pipelines for large-scale data ingestion, transformation, and enrichment.
• Develop and optimise data infrastructure within the Microsoft Azure ecosystem.
• Improve data quality, reliability, and processing performance.
• Make high-quality data accessible to AI agents and product features.
• Collaborate with engineering and product teams to deliver robust, production-ready solutions.

Skills

Data pipelines

Azure ecosystem

Big data

Collaboration

Tools

Microsoft Azure data services

Azure Data Factory

Databricks

PySpark

Azure Data Lake Storage

SQL

Python

Job description

Collective.work is building the next generation of AI-powered recruitment. We are seeking a freelance Data Engineer to design, build, and maintain scalable data pipelines in the Microsoft Azure ecosystem from London.

You will work with Azure Data Factory, Databricks, PySpark, SQL, and Python to ingest, transform, and enrich data for AI agents and product features. This contract role starts ASAP, with the day rate to be agreed.