Lead Data Engineer

JLA Resourcing Ltd; Greater London; On-site; GBP 70,000 - 80,000; Full time

Job summary

A leading UK resourcing firm is seeking a Lead Data Engineer to join their Basingstoke office. The role involves driving the design, development, and governance of data pipelines while working closely with stakeholders in a hybrid cloud environment. Candidates should have a strong background in data architecture, Databricks, and experience in implementing AI and machine learning solutions. The ideal applicant will also possess mentoring experience as the team is expected to grow.

Benefits

Bonus

Benefits

Qualifications

• In-depth experience in modern data solution architecture design in a hybrid cloud environment.
• Experience in pipeline orchestration management.
• Strong experience with Databricks.

Responsibilities

• Manage and mentor existing team members while driving design and governance of data pipelines.
• Ensure data platform architecture supports availability and growth targets.
• Implement and manage technology alongside 3rd Party Data Partners.

Skills

Databricks

Data solution architecture design

Pipeline orchestration management

Mentoring/Leading

Machine learning and AI implementation

Tools

Azure

Purview

Unity Catalog

Monte Carlo

Fabric Monitoring

Log Analytics

Kafka

Azure Stream Analytics

Azure ML

PowerBI

Delta Live Tables

Job description

Lead Data Engineer - £70-80k + bonus + benefits - Basingstoke 3 days a week

The Opportunity

We are looking for a Lead Data Engineer with strong Databricks experience to join a Basingstoke based organisation who are investing heavily in their Digital Transformation Programme.

The Role

You’ll play a proactive role in the delivery of next-generation data platforms, will manage / mentor the existing person and drive the design, development and governance of the data pipelines. You’ll be working really closely with stakeholders across the technology function and within the business and will the availability, integrity and compliance of the systems. You’ll play a key role in the ownership of the core architecture / engineering across the new Azure Databricks ecosystem.

You’ll ensure that the data platform architecture supports availability and growth targets and that the platforms leverage advances in AI and Machine Learning capability.

They are currently working with a 3rd Party Data Partner who have recommended a number of improvements - you’ll work closely with them selecting, implementing and managing technology so it's a great opportunity to really make a difference.

The Person

Key to this is proactivity - they're really looking for someone who is always looking at "what's next" - are there new tools or functionality that will help the business move forward. Other key background / attributes include :

• In depth experience of modern data solution architecture design and delivery in a hybrid cloud environment but predominantly Azure / Databricks
• Experience of pipeline orchestration management
• Strong exprerience with Databricks
• Experience of implementing machine learning and AI
• Tooling such as Purview and Unity Catalog as well as the use of observability tools such as Monte Carlo, Fabric Monitoring and Log Analytics
• Mentoring / Leading / Management experience initially with a small team but with a view that this will grow
• Ideally skills in Delta Live Tables, Kafka, Azure Stream Analytics, Azure ML, PowerBI and Financial Modelling experience.