Data Engineer

Jobtailor; Greater London; On-site; GBP 90,000 - 120,000; Full time

Job summary

Jobtailor is seeking a data engineer to design, build and maintain ETL/ELT pipelines, ingest data from internal systems and external APIs, and develop scalable data processing workflows. You will ensure pipelines are reusable, fault-tolerant, and aligned with data models, collaborating with the Data Architect and Analysts to deliver curated datasets with high accuracy and availability.

Experience with Python, SQL and data warehousing is required; you will optimise performance and cost-efficiency

Qualifications

• 3-7+ years experience in data engineering, backend engineering, or similar roles.
• Strong programming skills (e.g. Python, SQL).
• Solid understanding of ETL/ELT processes and data pipeline design.
• Experience working with APIs and integrating distributed systems.
• Experience handling transactional or event-based data.
• Strong understanding of data transformation techniques, data warehousing concepts, data modelling fundamentals.
• Experience with data orchestration and workflow tools.
• Ability to build robust, fault-tolerant systems.
• Strong problem-solving skills with attention to detail and data accuracy.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines.
• Ingest data from internal systems and external APIs.
• Develop scalable data processing workflows (batch and/or real-time).
• Ensure pipelines are reusable, efficient, and maintainable.
• Integrate data from various sources, including transactional flows.
• Handle complex data scenarios such as event sequencing, idempotency and duplicates.
• Ensure consistency between source systems and analytical datasets.
• Implement transformation logic aligned with architectural data models.
• Build and maintain structured data layers for analytics consumption.
• Collaborate with Data Architect on model implementation.
• Implement data validation, monitoring, and alerting mechanisms.
• Identify and resolve data inconsistencies or failures.
• Ensure data accuracy and availability.
• Optimise pipelines for performance and cost-efficiency.
• Support scaling of data infrastructure as volumes grow.
• Ensure low-latency data availability where required.
• Work with Data Architect, Analysts, and Manager.
• Support Analysts by providing curated datasets.
• Contribute to continuous improvement of data platform capabilities.

Skills

Python

SQL

ETL/ELT design

APIs

Data modelling

Data warehousing

Data orchestration

Fault-tolerant systems

Tools

APIs

Data Processing Workflows

Data Infrastructure

Job description

Responsibilities

• Design, build, and maintain ETL/ELT pipelines
• Ingest data from internal systems and external APIs
• Develop scalable data processing workflows (batch and/or real-time)
• Ensure pipelines are reusable, efficient, and maintainable
• Integrate data from various sources, including transactional flows
• Handle complex data scenarios such as: Event sequencing (e.g. bet -> resolve), Idempotency and duplicate handling, Partial or delayed data
• Ensure consistency between source systems and analytical datasets
• Implement transformation logic aligned with architectural data models
• Build and maintain structured data layers for analytics consumption
• Collaborate closely with the Data Architect on model implementation
• Implement data validation, monitoring, and alerting mechanisms
• Identify and resolve data inconsistencies or failures
• Ensure high levels of data accuracy and availability
• Optimise pipelines for performance and cost-efficiency
• Support scaling of data infrastructure as volumes grow
• Ensure low-latency data availability where required
• Work closely with Data Architect, Analysts, and Manager
• Support Analysts by ensuring availability of curated datasets
• Contribute to continuous improvement of data platform capabilities

Requirements

• 3-7+ years experience in data engineering, backend engineering, or similar roles
• Strong programming skills (e.g. Python, SQL)
• Solid understanding of ETL/ELT processes and data pipeline design
• Experience working with APIs and integrating distributed systems
• Experience handling transactional or event-based data
• Strong understanding of: Data transformation techniques, Data warehousing concepts, Data modelling fundamentals
• Experience with data orchestration and workflow tools
• Ability to build robust, fault-tolerant systems
• Strong problem-solving skills with attention to detail and data accuracy.

Core Competencies

Demonstrates expertise in designing and maintaining ETL/ELT pipelines, ensuring data accuracy and availability while optimizing performance and cost-efficiency. Proficient in data transformation techniques and integrating various data sources to support analytics.

Tools & Technologies

• APIs
• Data Processing Workflows
• Data Infrastructure