Principal Data Engineer - Azure Databricks (Unity Catalog)

Datatech; Cwmsyfiog; Hybrid; GBP 70,000 - 85,000; Full time

Job summary

A data-for-good organisation in the UK is seeking a Principal Data Engineer to help build and lead a modern data engineering function. The ideal candidate will have expertise in Azure Databricks and Python, responsible for implementing scalable data pipelines and deploying AI models. The position is hybrid, with a competitive salary range of £70,000-£85,000.

Qualifications

• Essential experience with Azure Databricks, including Unity Catalog.
• Knowledge of modern ELT workflows.
• Ability to communicate complex data concepts effectively.

Responsibilities

• Build and maintain scalable data pipelines for analytics and AI use cases.
• Lead the deployment of production-ready AI models.
• Support the development of Python coding standards.

Skills

Azure Databricks

Python (PySpark preferred)

SQL

Data governance

Collaboration and communication

Job description

Your shortlisted jobs will appear here. To view your shortlist: Login Or Register

Date Added: YESTERDAY

Principal Data Engineer - Azure Databricks (Unity Catalog)

Location: Bristol - Hybrid - 2 days a week in the office

Salary: £70,000-£85,000 Negotiable Dependent on Experience - 12 month Fixed Term Contract

Job Ref: J12997

This role is with a data-for-good organisation that is building a modern Analytics and Data Engineering function. They are looking for a Principal Data Engineer to support and lead key elements of this transformation.

This is a fantastic opportunity for someone who enjoys building data platforms from the ground up, mentoring teams, and deploying cutting-edge solutions in a collaborative environment.

The successful candidate will be responsible for designing and implementing scalable, reusable data pipelines to support analytics and AI projects. The organisation is transitioning to Azure and Databricks, and the Principal Data Engineer will play a crucial role in shaping this platform, upskilling the data team, and embedding best practices. They will also lead the deployment of AI models into production, ensuring monitoring, alerting, and pipeline reliability.

Key Responsibilities:

1. Build and maintain scalable data pipelines for analytics and AI use cases.
2. Lead the deployment of production-ready AI models with end-to-end monitoring.
3. Support the development of Python coding standards and engineering best practices.
4. Collaborate with IT and data teams to enhance the platform and capabilities.

Additionally, the role involves improving data governance and quality, supporting data model development, and fostering a culture of innovation and evidence-based decision-making.

Experience Required:

• Essential experience with Azure Databricks, including Unity Catalog, Python (PySpark preferred), and SQL.
• Knowledge of modern ELT workflows, focusing on Extract and Load stages.
• Ability to communicate complex data concepts to technical and non-technical teams.
• Adaptability, collaboration, problem-solving skills, and a focus on process improvement.

Note: Candidates must have the right to live and work in the UK; sponsorship is not available.

This high-impact role offers the chance to shape a new team, influence data strategy, and deliver tangible outcomes to support the organisation. If you're a capable Senior Data Engineer ready for a meaningful challenge, get in touch today.