Hybrid Data Engineer - Databricks & AWS (Belfast)

VanRath; Belfast City District; Hybrid; GBP 55,000 - 75,000; Full time

Benefits offered by this job

Flexible working

Hybrid/remote options

Pension

Healthcare

Job summary

VanRath in Belfast is seeking a Data Engineer to join the Data & Analytics team. The role is hybrid, focusing on building scalable data pipelines on Databricks and AWS, with Python/PySpark expertise.

You will design data architectures, work with data warehousing concepts and contribute to analytics and AI initiatives, while collaborating with stakeholders in a large-scale environment.

Qualifications

• 2+ years' experience in a Data Engineering or similar role.
• Strong experience with Python, PySpark and/or Databricks.
• Good understanding of Delta Lake, data warehousing and big data concepts.

Responsibilities

• Develop and maintain scalable ETL processes using Databricks and AWS Glue.
• Build automated data pipelines across cloud, on-premise, API and streaming sources.
• Develop data architectures aligned with business requirements.
• Build and maintain data models for BI and analytics.
• Work with Delta Lake and modern data warehousing technologies.
• Maintain Databricks notebooks, jobs and workflows.
• Optimise Spark jobs and data workflows for performance and scalability.
• Monitor, troubleshoot and improve data pipeline performance.
• Ensure data quality, integrity, security and compliance.
• Support wider analytics, AI and machine learning initiatives.
• Contribute to continuous improvement and knowledge sharing across the Data team.

Skills

Python

PySpark

Databricks

AWS

Data Warehousing

Education

Relevant degree or technical qualification

Tools

Delta Lake

Spark

Job description

VanRath in Belfast is seeking a Data Engineer to join the Data & Analytics team. The role is hybrid, focusing on building scalable data pipelines on Databricks and AWS, with Python/PySpark expertise.

You will design data architectures, work with data warehousing concepts and contribute to analytics and AI initiatives, while collaborating with stakeholders in a large-scale environment.