Databricks Data Engineer: Platform Modernisation

STARK Group; Huddersfield; On-site; GBP 60,000 - 90,000; Full time

Job summary

STARK Group is transforming its data platform and engineering capability to support analytics, self-service reporting, AI and data-driven decision making. We are hiring a hands-on Data Engineer to join our Data Engineering team and help build, improve and support trusted data products that underpin decision making across STARK UK.

You will migrate from Informatica and Azure Synapse toward a Databricks-centric platform, delivering Bronze-to-Platinum data layers and curated datasets for reporting

Qualifications

• Hands-on experience in a Data Engineering role.
• Strong hands-on Databricks experience building data pipelines and transformations.
• Strong Python skills for data engineering workloads.
• Hands-on Azure Synapse experience and the ability to migrate existing Synapse-based data solutions.
• Strong SQL, data modelling and transformation skills.
• Experience with cloud data storage and modern data platform concepts.
• Clear communication with technical colleagues, analysts and business stakeholders.

Responsibilities

• Design, build, test and maintain scalable data pipelines and data products using Databricks, Python, SQL and Azure data technologies.
• Ingest, transform and curate data from operational systems, files, APIs and other internal or external sources.
• Build and maintain data assets across Bronze, Silver, Gold and Platinum layers, following agreed architecture and engineering standards.
• Develop reusable, business-ready datasets that support analytics, reporting, semantic models and self-service use cases.
• Apply appropriate data modelling and transformation techniques to produce clear, performant and maintainable data structures.
• Ensure solutions are documented, version-controlled, testable and supportable.
• Contribute to the transition from Informatica and Azure Synapse-based solutions to Databricks, including analysis, migration, validation and decommissioning activity.
• Use existing Synapse knowledge to understand current pipelines and architecture, identify dependencies and support safe migration to the target platform.
• Refactor or rebuild legacy data processes where appropriate, improving scalability, maintainability and operational resilience.
• Work with the Lead Data Engineer, Architecture and delivery partners to adopt agreed Databricks patterns and standards.
• Support effective knowledge transfer from System Integrators and other third parties, building internal team capability and reducing long-term dependency.
• Build data quality controls, automated tests, monitoring and alerting into engineering solutions.
• Follow DataOps, CI/CD, source-control and deployment practices, contributing to their continued improvement.
• Investigate and resolve data pipeline failures, quality issues and performance problems, including appropriate operational support for production data services.
• Identify technical debt and manual processes and contribute practical improvements that reduce risk and support effort.
• Maintain clear technical documentation, lineage and operational knowledge so that solutions can be understood and supported across the team.
• Ensure engineering work complies with security, governance and data management standards.
• Work with Data Analysts and business stakeholders to understand data requirements and translate them into well-designed engineering solutions.
• Partner with other Data Engineers through code reviews, problem solving and knowledge sharing.
• Collaborate with Architecture, Insight & Analytics, Testing and Technology teams to deliver joined-up solutions.
• Communicate delivery progress, risks, dependencies and technical issues clearly to the Lead Data Engineer and relevant stakeholders.
• Develop an understanding of STARK UK business processes and data domains so that engineering decisions reflect business context as well as technical requirements.

Skills

Databricks

Python

Azure Synapse

SQL

Data modelling

Data pipelines

Data quality

CI/CD

DataOps

Cloud storage

Communication

Tools

Informatica

Azure Synapse

Delta Lake

Power BI

CI/CD tooling

Job description

STARK Group is transforming its data platform and engineering capability to support analytics, self-service reporting, AI and data-driven decision making. We are hiring a hands-on Data Engineer to join our Data Engineering team and help build, improve and support trusted data products that underpin decision making across STARK UK.

You will migrate from Informatica and Azure Synapse toward a Databricks-centric platform, delivering Bronze-to-Platinum data layers and curated datasets for reporting