Lead Data Engineer - Commodities Trading | London, UK | Hybrid

Are you a Data Engineering leader looking for your next big challenge? I'm searching for a Lead Data Engineer to spearhead an exciting global trade surveillance initiative for a prestigious client within the commodity trading sector. This is an opportunity to shape the future of compliance-driven data solutions, leveraging cutting-edge technologies like Azure Synapse and DataBricks to optimise data ingestion and processing.

Role Responsibilities:

• Leading a team of data engineers to design and implement scalable data pipelines for trade surveillance.
• Owning the global roadmap for trade surveillance rollout, ensuring seamless integration with business initiatives.
• Driving the evolution of the data lakehouse architecture, enhancing speed, efficiency, and quality.
• Acting as the key technical point of contact, influencing strategic decision-making.
• Fostering a high-performance, agile team culture that thrives on innovation.

Your background:

• Proven experience leading high-performing data engineering teams in a fast-paced environment.
• Expertise in Python, PySpark, SQL, Synapse, DataBricks, and Azure Data Factory (ADF).
• Strong knowledge of CI/CD pipelines, Git, Jenkins, Docker, and test automation (PyTest).
• Hands-on experience with Big Data architectures, Data Lakehousing, and real-time data processing.
• Exceptional communication skills—translating complex technical challenges to both technical and non-technical stakeholders.
• Bonus: Experience in commodity trading operations (Oil, Gas, Shipping, etc.).