Data Engineer - Azure, Databricks & Pipelines

Independent Vetcare Limited; West of England; Hybrid; GBP 37,000 - 77,000; Full time

Benefits offered by this job

Healthcare Cash Plan

Cycle to Work scheme

Green Cars salary sacrifice scheme

Staff pet care discounts

Retailer discounts

Job summary

IVC Evidensia is seeking a skilled data engineer to build and optimise data pipelines using PySpark within a Microsoft Azure ecosystem. You will work as a technical authority, collaborating with SMEs and stakeholders to deliver scalable data solutions across the group data platform.

The role emphasizes data modelling, secure development, and an international, collaborative mindset, with a hybrid base in Keynsham, Bristol. Strong BI involvement with Power BI is advantageous.

Qualifications

• Hands-on Databricks experience with Delta Lake, Unity Catalog and Declarative Automation Bundles.
• Proficient in writing and optimising PySpark for data transformation and processing.
• Solid experience in Azure ecosystem: ADF, ADLS and Azure DevOps.
• Experience with Azure SQL Server as data warehouse foundation.
• Experience designing pipelines using REST APIs, JSON, and ELT.
• Comfort with relational and non-relational data modelling principles.
• Exposure to Power BI, desirable for BI solutions.
• Excellent communication and stakeholder engagement skills.
• Agile mindset with secure development practices and non-functional requirements like scalability and access management.
• International mindset with collaborative approach.

Responsibilities

• Take ownership of building, optimising, and maintaining data pipelines using PySpark.
• Orchestrate feeds from source systems to the group data platform.
• Act as technical authority for design and development of data technologies.
• Collaborate with business to deliver real-world data solutions.
• Engage with SMEs to break requirements into deliverables.
• Work with Data Architect to shape data management strategy.
• Interface between IT and functions like Finance and Marketing.

Skills

Databricks

Delta Lake

Unity Catalog

Declarative Automation Bundles

PySpark

Azure DevOps

Azure Data Factory

Azure Data Lake Storage

Azure SQL Server

REST APIs

JSON

Data Modelling

Power BI

Communication

Agile

Tools

Azure Data Factory

Azure Data Lake Storage

Power BI

Job description

IVC Evidensia is seeking a skilled data engineer to build and optimise data pipelines using PySpark within a Microsoft Azure ecosystem. You will work as a technical authority, collaborating with SMEs and stakeholders to deliver scalable data solutions across the group data platform.

The role emphasizes data modelling, secure development, and an international, collaborative mindset, with a hybrid base in Keynsham, Bristol. Strong BI involvement with Power BI is advantageous.