Senior Data Engineer – Real-Time Streaming Architect

JobCubby; Swindon; On-site; GBP 93,000 - 135,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) with company match

Tuition reimbursement

Paid time off + holidays

Volunteer time

Job summary

JobCubby in Swindon is seeking a senior data engineer to design and implement scalable data pipelines and real-time streaming solutions. You will coordinate across teams to translate complex requirements into robust data platforms.

You should bring 6-8+ years of data engineering experience with Kafka, Confluent Cloud, AWS Kinesis or Pulsar, and strong Python, PySpark, and SQL skills, plus Docker, Kubernetes, Terraform, and CI/CD for microservices.

Qualifications

• Bachelor’s degree in computer science or related field.
• 6-8+ years of data engineering experience with enterprise scale platforms.
• Experience with Apache Kafka, Confluent Cloud, AWS Kinesis, or Apache Pulsar.
• Advanced proficiency in Python, PySpark, and SQL.
• Production experience using Flink, Kafka Streams, or Spark Structured Streaming.
• Strong experience with AWS and modern data platforms.
• Deep expertise with CDC tools such as Debezium and Kafka Connect.
• Sinking streaming events into lakehouse formats like Iceberg/Delta Lake and cloud warehouses like Snowflake/BigQuery.
• Foundational knowledge of Docker, Kubernetes, Terraform, and CI/CD for event-driven microservices.
• Experience leading initiatives and mentoring engineers.
• Strong communication and Agile experience.

Responsibilities

• Lead design and implementation of advanced data pipelines, ingestion frameworks, and real-time streaming solutions.
• Build, tune, and maintain scalable Kafka producers and consumers for sub-second latency and high availability.
• Configure CDC solutions across relational and NoSQL sources into Kafka topics.
• Develop stream-processing logic using Flink, Spark Streaming, Kafka Streams, or dbt-mesh architectures.
• Enforce schema evolution policies using Confluent Schema Registry and ensure data quality and idempotency.
• Mentor engineers and lead code reviews to uphold engineering standards.
• Collaborate with architects to align solutions with enterprise direction.
• Drive automation, efficiency, and platform capability improvements.

Skills

Communication

Leadership

Mentoring

Problem-solving

Agile

Education

Bachelor's degree in CS or related

Tools

Apache Kafka

Confluent Cloud

AWS Kinesis

Apache Pulsar

PySpark

SQL

Flink

Kafka Streams

dbt-mesh

Schema Registry

Job description

JobCubby in Swindon is seeking a senior data engineer to design and implement scalable data pipelines and real-time streaming solutions. You will coordinate across teams to translate complex requirements into robust data platforms.

You should bring 6-8+ years of data engineering experience with Kafka, Confluent Cloud, AWS Kinesis or Pulsar, and strong Python, PySpark, and SQL skills, plus Docker, Kubernetes, Terraform, and CI/CD for microservices.