Data Engineer - Operational Data

FRASERS GROUP PLC t/a Sports Direct International; United Kingdom; On-site; GBP 60,000 - 80,000; Full time

Job summary

A leading sports retail company in the United Kingdom is seeking a Mid-Senior Data Engineer to join its Data Engineering team. This role focuses on designing and maintaining scalable data pipelines using Databricks and Azure data services. The ideal candidate has expertise in real-time data processing and strong skills in Spark and Python. The company offers a dynamic work environment with ample opportunity for professional growth and personal recognition.

Benefits

Employee recognition program

Gym memberships and free classes

Mental health support services

Qualifications

• Extensive hands-on experience with the Databricks Lakehouse Platform.
• Proficiency in Spark programming (PySpark and/or Scala) and expert-level SQL skills.
• Demonstrable experience in building and managing stream and micro-batch processing pipelines.

Responsibilities

• Design and maintain robust data pipelines using Databricks and Spark.
• Implement Change Data Capture mechanisms for near real-time data.
• Collaborate with cross-functional teams and mentor junior engineers.

Skills

Databricks expertise

Spark programming

Python/SQL skills

Real-time data processing

Data modelling concepts

Problem-solving skills

Tools

Databricks Lakehouse Platform

Azure data services

GCP and BigQuery

Job description

Company Description

At Frasers Group we’re rethinking retail. Through digital innovation and unique store experiences, we’re serving our consumers with the world’s best sports, premium and luxury brands globally. As a leader in the industry, we’re elevating the retail experience for our consumers through our collection of established brands, including Sports Direct, FLANNELS, USC, Frasers, and GAME.

Why join us?

Our vision - we are building the worlds most admired and compelling brand ecosystem

Our purpose – we are elevating the lives of the many with access to the world’s best brands and experiences

At Frasers Group, we fear less and do more. Our people are forward thinkers who are driven to operate outside of their comfort zone to change the future of retail, embracing challenges along the way. The potential to elevate your career is massive, the experience unrivalled. To be able to make the most of it you need to live and breathe our principles:

• Think without limits - Think fast, think fearlessly, and take the team with you
• Own it and back yourself - Own the basics, own your role and own the results
• Be relevant - Relevant to our people, our partners and the planet

Are you ready to join the Fearless?

Job Description

We are currently looking for a Mid‑Senior Data Engineer to join us in our growing Data Engineering team; to help develop, maintain, support, and integrate our growing number of data systems. You will be instrumental in designing, building, and maintaining robust and scalable data pipelines that power our operational subscribers and analytical platforms.

The right candidate will have a wealth of knowledge in the data world with a strong focus on Databricks, and will be keen to expand upon their existing knowledge, learning new technologies along the way as well as supporting both future and legacy technologies and processes.

You will be coding, testing, and documenting new or modified data systems; creating scalable, repeatable, secure pipelines and applications for both operational data and analytics, both internally and externally to the business. You will grow our capabilities, solving new data problems and challenges every day.

Key Responsibilities

• Design, Build, and Optimise Real‑Time Data Pipelines: Develop and maintain robust and scalable stream and micro‑batch data pipelines using Databricks, Spark (PySpark/SQL), and Delta Live Tables.
• Implement Change Data Capture (CDC): Implement efficient CDC mechanisms to capture and process data changes from various source systems in near real‑time.
• Master the Delta Lake: Leverage the full capabilities of Delta Lake, including ACID transactions, time travel, and schema evolution, to ensure data quality and reliability.
• Champion Data Governance with Unity Catalog: Implement and manage data governance policies, data lineage, and fine‑grained access control using Databricks Unity Catalog.
• Enable Secure Data Sharing with Delta Sharing: Design and implement secure and governed data sharing solutions to distribute data to both internal and external consumers without data replication.
• Integrate with Web Services and APIs: Develop and manage integrations to push operational data to key external services, as well as internal APIs.
• Azure Data Ecosystem: Work extensively with core Azure data services, including Azure Data Lake Storage (ADLS) Gen2, Azure Functions, Azure Event Hubs, and CI/CD.
• Data Modelling and Warehousing: Apply strong data modelling principles to design and implement logical and physical data models for our analytical and operational data stores.
• Monitoring and Performance Tuning: Proactively monitor data pipeline performance, identify bottlenecks, and implement optimisations to ensure low latency and high throughput.
• Collaboration and Mentorship: Collaborate with cross-functional teams, including software engineers, data scientists, and product managers, and mentor junior data engineers.

Qualifications

What We’re Looking For

• Proven Databricks Expertise: Extensive hands‑on experience with the Databricks Lakehouse Platform is essential.
• Strong Spark and Python/SQL Skills: Proficiency in Spark programming (PySpark and/or Scala) and expert‑level SQL skills.
• Real‑Time Data Processing: Demonstrable experience in building and managing stream and micro‑batch processing pipelines using technologies like Spark Structured Streaming or Delta Live Tables.
• Deep Understanding of Delta Lake Concepts: Thorough knowledge of Delta Lake architecture and features (ACID transactions, time travel, optimisation techniques).
• Experience with Databricks Advanced Features: Practical experience with Change Data Capture (CDC), Unity Catalog for data governance, and Delta Sharing for secure data collaboration.
• Web Service and API Integration: A proven track record of integrating data pipelines with external web services and REST APIs.
• Solid Azure Experience: Strong experience with core Azure data services (ADLS Gen2, Event Hubs, Azure Functions).
• Data Modelling and Warehousing Fundamentals: A strong understanding of data modelling concepts (e.g., Kimball, Inmon) and experience with data warehousing principles.
• CI/CD and DevOps Mindset: Experience with CI/CD pipelines for data engineering workloads using tools like Git.
• Excellent Problem‑Solving and Communication Skills: The ability to troubleshoot complex data issues and communicate technical concepts effectively to both technical and non‑technical audiences.

Desirable

• GCP and BigQuery Knowledge: Utilize your experience with GCP and BigQuery for analytical data workloads and to ensure seamless interoperability within our multi‑cloud strategy.

Additional Information

Along with your benefits package we also offer a wide range of perks for our colleagues:

Reward, Recognition and Opportunities

Frasers Champion- Our employees are at the heart of our business and we ensure individuals are recognised every single month for their hard work. Frasers Champion is a peer nominated scheme where 8 winners will receive double their pay for a month where they have thought without limits, owned it or been relevant.

Retail Reconnect – In order to build the planets most admired and compelling brand ecosystem, all employees must understand our business, product and customers. Each financial year, Head Office employees will gain insights by spending two days in one of our stores or the Warehouse. The goal is to learn how the work you do impacts our teams on the frontline, and to bring ideas back to the office which will improve how we work.

Employee Welfare

Frasers Fit – Our Everlast Gyms Team are on a mission to make our workforce the best, and fittest on the planet! We run free gym classes for employees as well as discounted memberships to our clubs. Frasers Fit is our wellbeing programme which aims to support and improve colleagues Physical, Financial & Mental wellbeing. The app is accessible for every employee and includes training, nutrition and lifestyle advice- all completely free.

Retail Trust – We know that its not just about physical health, mental wellness is equally important which is why all of our employees get free access and support from the Retail Trust charity. This includes a 24 hour wellbeing helpline, wellness hub, counselling and financial/legal support.

What’s next?

Our Recruitment Team will be reviewing applications and all candidates will receive a response, whether you are successful or unsuccessful. Shortlisted applicants may be asked to confirm a few key details before being booked in for a first stage interview with the Recruiter- this will be behaviourally focussed and centred around how you align with our Culture and Values. If successful we anticipate two further interview stages with the Hiring Manager/wider team which will be more technically focussed and could include a presentation/task so we can see your skills in action.