Data Engineer

Auxo Talent; Greater London; On-site; GBP 60,000 - 90,000; Full time

Job summary

Auxo Talent is seeking an experienced Data Engineer to support a major trading automation programme. Remote UK with occasional travel to London for stakeholder meetings and workshops.

You will design and build data pipelines on Databricks and Azure, delivering scalable, governance-backed data foundations for AI-powered trading workflows and analytics. The role demands strong Python and SQL, Delta Lake expertise, batch and streaming pipelines, and collaboration with AI engineers, trading teams

Qualifications

• Design scalable data pipelines with Databricks and Azure.
• Strong Python and SQL skills.
• Experience with Delta Lake and lakehouse architecture.
• Experience with CI/CD, Git and IaC.
• Knowledge of data modelling, testing, governance and security.

Responsibilities

• Design, build and maintain scalable data pipelines using Databricks and Azure.
• Develop ingestion and transformation processes for structured and unstructured data.
• Build data products to support trading workflows and Generative AI applications.
• Implement data architectures using Delta Lake and lakehouse principles.
• Apply data quality, governance, lineage and security controls.
• Optimise pipelines and datasets for performance and reliability.
• Collaborate with AI engineers, trading teams and stakeholders.
• Support automated deployment, monitoring and operational management.

Skills

Databricks

Azure

Python

SQL

Data pipelines

Delta Lake

Spark

CI/CD

Git

Infrastructure as Code

Tools

Databricks Unity Catalog

MLflow

Job description

Location: Remote, UK, with travel to London 1-2 times per quarter

Duration: 6 months initially

Start Date: October 2026

We are looking for an experienced Data Engineer to support a major Trading Automation programme.

You will design and build modern data pipelines and platform capabilities using Databricks and Azure, providing the trusted and scalable data foundations required for AI-driven trading automation.

Key Responsibilities

• Design, build and maintain scalable data pipelines using Databricks and Azure.
• Develop ingestion and transformation processes for structured and unstructured data.
• Build reliable data products to support trading workflows and Generative AI applications.
• Implement data architectures using Delta Lake and lakehouse principles.
• Apply data quality, governance, lineage and security controls.
• Optimise pipelines and datasets for performance, reliability and scalability.
• Collaborate with AI engineers, trading teams and business stakeholders.
• Support automated deployment, monitoring and operational management of data workloads.
• Advanced experience with Databricks, Apache Spark and PySpark.
• Strong knowledge of Azure data services.
• Experience building batch and streaming data pipelines.
• Strong Python and SQL skills.
• Experience with Delta Lake and lakehouse architecture.
• Knowledge of data modelling, testing, quality and governance.
• Experience with CI/CD, Git and Infrastructure as Code.
• Comfortable working independently in a remote, client-facing environment.
• Experience within Energy Trading, commodities, Capital Markets or Financial Services.
• Experience supporting AI, machine learning or Generative AI workloads.
• Knowledge of Databricks Unity Catalog and MLflow.