Data Engineer - Full-Stack Data Products & Pipelines

Catalyst; Newcastle upon Tyne; On-site; GBP 65,000 - 95,000; Full time

Job summary

Catalyst, a Newcastle upon Tyne based data engineering team, is seeking a Data Engineer to build and maintain data pipelines and platforms powering analytics and AI-driven products.

You will combine full-stack or backend software engineering with modern data engineering, working across cloud infrastructure and data platforms to enable reliable data access and innovative analytics solutions.

Qualifications

• Expertise in designing and building data intensive applications and pipelines, with an open mindset on choice of tools.
• Strong proficiency in Python and SQL, with a focus on clean, performant code to process and manage large volumes of data.
• Experience designing and implementing complex data-matching, reconciliation, and entity-resolution solutions across multiple data sources.
• Distributed computing for data intensive work such as Spark and Databricks.
• Cloud platforms including AWS, Azure and GCP.
• Orchestration tools like Airflow.
• Modern DevOps practices including version control, CI/CD, and infrastructure as code using Terraform or Kubernetes.
• Experience building scalable backend applications.
• Understanding of distributed systems and software architecture.
• Ability to test and troubleshoot data products with a track record of improving data reliability and quality.
• Excellent communication skills and the ability to work effectively and openly within a team-based environment.
• Self-motivated, detail-oriented, and eager to learn new technologies and methodologies.

Responsibilities

• Develop, and maintain production-grade applications and APIs.
• Develop reusable, well-tested code following engineering best practices.
• Architecture: Participate in architecture and technical design discussions.
• Testing: Write automated unit, integration, and end-to-end tests.
• Data Pipeline Development: Design and develop sophisticated data pipelines to import data from internal and external sources and seamlessly integrate into data platforms.
• Maintenance & Data Reliability: Ensure systems are well-maintained, enabling uninterrupted access to crucial data. Continuously verify data quality and integrity to support decision making.
• Continuous Improvement & Collaboration: Collaborate with internal and external stakeholders to streamline data accessibility and usability. Drive innovation by adopting advanced analytics technologies and methodologies.
• Cloud infrastructure: Deploy cloud-native solutions using Infrastructure as Code and collaborate on cloud architecture and optimisation

Skills

Python

SQL

Data engineering

Distributed systems

Communication

Education

2:1 or 1st in Computing Science/Maths/Physics

Tools

Spark

Databricks

Airflow

Terraform

Kubernetes

CI/CD

AWS

Azure

GCP

Job description

Catalyst, a Newcastle upon Tyne based data engineering team, is seeking a Data Engineer to build and maintain data pipelines and platforms powering analytics and AI-driven products.

You will combine full-stack or backend software engineering with modern data engineering, working across cloud infrastructure and data platforms to enable reliable data access and innovative analytics solutions.