Data Engineer

Service Care Solutions; United Kingdom; Hybrid; GBP 72,000 - 88,000; Full time

Job summary

Service Care is seeking an experienced Data Engineer to join its Data, Analytics & Reporting team in the UK. You’ll design, build and optimise data pipelines on a Databricks platform hosted on AWS, delivering trusted data for analytics, reporting and data science.

You'll work with Data Scientists and Analysts to transform data from clinical, operational and external sources, implement data governance, and contribute to machine learning workflows while maintaining data quality and security.

Qualifications

• Proven experience as Data Engineer or similar role.
• Strong SQL and data modelling skills.
• Proficient in Python for data transformation.
• Experience with Databricks, Delta Lake and Unity Catalog.
• CI/CD pipelines and Git version control.
• Knowledge of GDPR and data governance.

Responsibilities

• Design, build, test and optimise ETL/ELT pipelines on Databricks platform on AWS.
• Ingest data from clinical, operational and external sources.
• Develop SQL and PySpark transformations across Bronze, Silver and Gold layers.
• Implement automated data quality checks and monitoring.
• Collaborate with Data Scientists on datasets for ML/NLP.
• Support deployment via Git, CI/CD and IaC.

Skills

Strong SQL

PySpark

Databricks

REST APIs

CI/CD

Python

Data modelling

Git

Tools

Delta Lake

dbt

Unity Catalog

Terraform

Job description

Data Engineer Salary: Up to £80,000 FTE Location: Home-based, with occasional travel to meetings as required Hours: 40 hours per week Reporting to: Lead Data Engineer

The Opportunity

We are working with a leading UK-based online mental healthcare provider that is looking to appoint an experienced Data Engineer to join its growing Data, Analytics & Reporting team. This is an exciting opportunity to join an organisation using innovative technology and AI-enabled platforms to improve access to high-quality mental healthcare across the UK.

As Data Engineer, you will play a key role in developing, maintaining and optimising the organisation's data infrastructure, ensuring reliable, secure and high-quality data is available to support analytics, reporting and data science. Working closely with the Lead Data Engineer, Data Scientists, Analysts and wider clinical and operational teams, you will help transform data from multiple clinical, operational and third-party systems into trusted, well-governed datasets. This is a hands-on engineering role, offering the opportunity to contribute to the development of a modern Databricks Analytics Platform and help shape the organisation's wider data capabilities.

Key Responsibilities

• Design, build, test and optimise robust ETL/ELT data pipelines within a Databricks Analytics Platform hosted on AWS.
• Develop and maintain data ingestion processes from clinical, operational, financial and external systems, including relational databases, REST APIs and file-based sources.
• Build and optimise SQL and PySpark transformations across Bronze, Silver and Gold data layers.
• Work with technologies including Databricks, Delta Lake, Lakeflow, dbt and Unity Catalog.
• Implement automated data quality checks, monitoring and alerting to ensure data accuracy, completeness and reliability.
• Manage and orchestrate workloads, troubleshoot pipeline failures and support production incident resolution.
• Support deployment processes across development, staging and production environments using Git, CI/CD and Infrastructure as Code principles.
• Apply data governance and security controls, including access permissions, data masking, pseudonymisation and data lineage.
• Monitor and optimise platform performance, resource usage and cloud infrastructure costs.
• Collaborate with Data Scientists to provide high-quality datasets for machine learning, NLP and predictive analytics.
• Support feature engineering, model training and inference workflows, including the use of MLflow.
• Work alongside Analysts to develop Gold-layer dimensional models and governed metrics for Power BI, Databricks AI/BI dashboards and Genie.
• Maintain clear technical documentation covering data models, schemas, transformations and data flows.
• Promote best practice in data engineering through code reviews, knowledge sharing, collaboration and continuous improvement.
• Ensure all data processes comply with relevant information governance, GDPR and NHS data security requirements.

About You

We are looking for a technically strong and motivated Data Engineer who has experience building and supporting production data pipelines and enjoys working collaboratively within a growing data team.

You will ideally have:

• Proven experience working as a Data Engineer or in a similar data engineering role.
• Strong SQL skills, including performance tuning, CTEs, window functions and data modelling.
• Good Python programming skills, particularly PySpark, for data transformation, API integrations and automation.
• Hands-on experience with Databricks, including Delta Lake, Spark and Unity Catalog, or a comparable cloud data platform such as Snowflake or AWS Redshift.
• Experience working with data ingestion techniques, including change data capture (CDC), REST API integrations and incremental loading.
• Understanding of data modelling approaches, including star schemas, medallion architecture and slowly changing dimensions.
• Experience with Git, version control and CI/CD pipelines.
• An understanding of Infrastructure as Code principles, ideally with exposure to Terraform or Declarative Automation Bundles.
• Knowledge of data governance and security, including access controls, data masking and pseudonymisation.
• An awareness of data science and machine learning workflows and how data engineering supports these processes.

Personal Attributes

Alongside your technical expertise, you will demonstrate:

• Strong analytical and problem-solving abilities.
• Excellent attention to detail.
• The ability to manage your own workload and prioritise competing demands.
• A collaborative approach and the ability to work effectively with technical and non-technical stakeholders.
• Strong written and verbal communication skills.
• A proactive attitude towards learning, development and continuous improvement.

Why Consider This Opportunity?

• Salary of up to £80,000 FTE.
• Fully home-based working arrangement.
• Opportunity to work with modern cloud data technologies and a Databricks-based analytics platform.
• Exposure to AI, machine learning and advanced data science projects.
• The chance to contribute to meaningful projects within the healthcare sector.
• Opportunity to work alongside an experienced and collaborative Data, Analytics & Reporting team.