Real-Time Data Platform Engineer (Python + AWS) Hybrid

Permutable; Greater London; Hybrid; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Equity options

Hybrid flexibility

Job summary

Permutable is hiring an experienced Data Platform Engineer to build real-time data pipelines and own the AWS infrastructure that runs them. You'll design Python services to ingest and transform large data volumes, while scaling S3, RDS/PostgreSQL, Lambda, ECS and EKS for fast, reliable delivery to models and clients.

You'll work in a small, fast-moving team from our London hub, with three days on-site and flexible remote work.

Qualifications

• Strong production Python experience with building scalable data pipelines.
• Hands-on AWS experience: S3, RDS/PostgreSQL, Lambda, ECS or EKS.
• Experience designing and maintaining data platforms and pipelines for real-time workloads.
• Proficiency in SQL, PostgreSQL, data modelling and large dataset processing.
• Familiarity with IaC (Pulumi/Terraform) and CI/CD pipelines.

Responsibilities

• Design and develop Python services and pipelines to ingest, transform and deliver large data volumes.
• Build and scale AWS data platform infrastructure across S3, RDS/PostgreSQL, Lambda, ECS and EKS.
• Develop event-driven services using messaging and queues for reliable workflows.
• Improve data quality with validation, deduplication, retries and recovery.
• Own storage, querying and data serving for research and client apps.
• Collaborate with AI/ML teams to provide data inputs and model outputs.
• Automate infrastructure and deployments with Pulumi and GitHub Actions.
• Monitor, diagnose and optimize pipeline performance and costs.

Skills

Python

AWS

Data pipelines

SQL

PostgreSQL

Kubernetes

Terraform

CI/CD

Linux

Pulumi

Education

Bachelor's degree in computer science or related field

Tools

Pulumi

GitHub Actions

Terraform

Job description

Permutable is hiring an experienced Data Platform Engineer to build real-time data pipelines and own the AWS infrastructure that runs them. You'll design Python services to ingest and transform large data volumes, while scaling S3, RDS/PostgreSQL, Lambda, ECS and EKS for fast, reliable delivery to models and clients.

You'll work in a small, fast-moving team from our London hub, with three days on-site and flexible remote work.