Senior Data Engineer

Retelligence; England; Hybrid; GBP 90,000 - 120,000; Full time

Job summary

A digital innovation organization is seeking a Senior Data Engineer located in London, with a hybrid working model. This role involves designing and maintaining high-performance data pipelines, ensuring data integration, and building optimized data models. Candidates should have extensive experience with Google Cloud Platform, strong Python skills, and familiarity with technologies like Kafka. The position is full-time at the mid-senior level, ideal for those thriving in a collaborative and fast-paced environment.

Qualifications

• Extensive experience architecting and managing real-time data pipelines.
• Deep experience with Google Cloud Platform and its tools.
• Strong proficiency in Python for data development.
• Strong data modeling, quality, security and governance practices.
• Experience with Airflow or Kubernetes for orchestration.

Responsibilities

• Lead the design and maintenance of data pipelines and infrastructure.
• Ensure seamless data flow by integrating diverse data sources.
• Build and optimize data models for analytical use cases.
• Ensure data quality, integrity and security across systems.
• Implement monitoring, logging, and alerting for reliability.
• Collaborate with technical and business teams to align data infrastructure with objectives.

Skills

Data Engineering

Google Cloud Platform (GCP)

Kafka

Python

SQL

Apache Airflow

Data quality

Airflow

Kubernetes

Tools

Apache Airflow

Kubernetes

Job description

This range is provided by Retelligence. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

Direct message the job poster from Retelligence

Job Title: Senior Data Engineer

Location / Working model: London | Hybrid

This is a critical technical position within a high-growth, forward-thinking organization that specializes in digital innovation. We require an experienced Senior Data Engineer to design, build, and maintain a robust, scalable, and high-performance system that is fundamental to the organization's operational and analytical success.

Core Responsibilities

• System Architecture & Development: Lead the design, development, and maintenance of scalable, high-performance, real-time data pipelines and core infrastructure components within the cloud environment.
• Data Integration: Ensure seamless, low-latency data flow by integrating and unifying diverse data sources across the organization.
• Data Modeling & Quality: Build and optimize robust data models specifically for complex querying and analytical use cases, and ensure data quality, integrity, and security across all systems.
• Reliability & Monitoring: Construct highly available, fault-tolerant systems for high-volume data ingestion and processing, implementing effective monitoring, logging, and alerting mechanisms.
• Performance Optimization: Continuously monitor, tune, and optimize pipeline scalability and performance to meet stringent throughput and latency standards.
• Cross-Functional Alignment: Partner directly with technical and business teams to ensure the data infrastructure directly supports strategic objectives.

Required Technical Expertise

• Production Experience: Extensive hands-on experience in Data Engineering, specifically a proven track record of successfully architecting, building, and managing production-grade real-time data pipelines across multiple large-scale initiatives.
• Cloud Platform Specialization: Mandatory deep, practical experience leveraging Google Cloud Platform (GCP) and its associated tools for the deployment and management of real-time data ingestion and processing services. (While AWS knowledge is valuable, GCP expertise is primary.)
• Distributed Streaming: Strong familiarity with distributed streaming technologies, such as Kafka or similar platforms.
• Programming & Scripting: Expert proficiency in Python for development, coupled with the ability to optimize and refactor data pipelines for improved performance and scalability.
• Database Expertise: Advanced knowledge of SQL is mandatory, alongside a strong understanding of various database technologies (NoSQL, time-series databases) and deep competence in data modeling principles.
• Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes).

The Environment

We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving in a fast-paced environment. Exceptional written and verbal communication skills in English are required for effective technical leadership and documentation.

Seniority level: Mid-Senior level

Employment type: Full-time

Industries: Staffing and Recruiting, IT Services and IT Consulting

Referrals increase your chances of interviewing at Retelligence by 2x