Senior Data Engineer (Python & AWS)

Morgan McKinley; Philippines; Hybrid; PHP 1,500,000 - 2,100,000; Full time

Job summary

Morgan McKinley seeks a Senior Data Engineer to lead strategic data pipelines and calculation engines using Python and AWS serverless tech. You will partner with Risk and Technology teams in London to deliver high-throughput ingestion, transformation, and real-time risk capabilities.

The role emphasizes AI-assisted development, TDD, and robust microservice design. You will work across data, messaging, and cloud platforms with a strong focus on production-grade data systems.

Qualifications

• 7+ years of software/data engineering experience with Python.
• Experience building real-time and batch data pipelines.
• Hands-on with serverless cloud architectures (AWS/Azure).
• Familiarity with microservices, messaging, and data storage (SQL/NoSQL).
• Proficient in TDD, CI/CD, and modern software engineering practices.
• Aware of financial data concepts and risk domain is a plus.

Responsibilities

• Design, build, and operate scalable real-time and batch data pipelines.
• Develop cloud-native, event-driven services using AWS Lambda and Step Functions.
• Leverage AI coding tools to accelerate development and testing.
• Define data models, APIs, and integration patterns for distributed systems.
• Implement end-to-end automation, tests, and robust CI/CD pipelines.

Skills

Python

Event-Driven

Microservices

TDD

CI/CD

Data Pipelines

Problem Solving

Cloud Concepts

Tools

AWS Lambda

AWS Step Functions

S3

DynamoDB

Kafka

PostgreSQL

Terraform

Docker

Job description

Shift: Mid shift aligned with UK/EMEA timezone

Employment Type: Full-Time, Permanent

Onboarding: First month fully onsite; then hybrid 3 days onsite per week

Role Overview

As a Senior Data Engineer, you will lead the development of strategic, event-driven data and calculation pipelines using Python and AWS Serverless technologies. Working alongside our Risk and Technology teams in London, your primary mandate will be constructing high-throughput ingestion, transformation, and calculation engines that power real-time risk capabilities.

This role heavily embraces modern software engineering practices: we operate with an AI-first development culture (using coding agents daily), prioritize Test-Driven Development (TDD), and emphasize robust messaging and microservice design. You do not need to be a C# or frontend developer; while our legacy platform uses C# .NET and web UIs, your role is focused squarely on building the modern Python cloud infrastructure (reading legacy code only when necessary to translate business logic).

Role Responsibilities

• Data Pipeline Engineering: Design, build, and operate scalable real-time and batch data pipelines—handling ingestion, transformation, enrichment, and orchestration from heterogeneous data feeds.
• Serverless Architecture: Develop cloud-native, event-driven services using serverless compute (AWS Lambda / Azure Functions) and orchestration tools (AWS Step Functions / state machines).
• AI-Enabled Development: Daily usage of AI coding tools (Claude Code, Amazon Q, GitHub Copilot) to accelerate development velocity, generate comprehensive test suites, and maintain living documentation.
• Messaging & Integration: Design data models, API contracts, and integration patterns for distributed systems consuming event streams (Kafka, Solace) and relational/non-relational storage (PostgreSQL, DynamoDB, S3).
• Quality & Automation: Implement TDD practices, end-to-end automated testing (unit, integration, contract), and maintain robust CI/CD deployment pipelines.

Experience / Competences

• Deep Python Expertise: 7+ years of software development experience with hands-on mastery of production-grade Python for complex backend or data calculation workloads.
• AWS / Cloud Serverless: Proven track record building and deploying serverless applications (AWS Lambda, Step Functions, S3, EventBridge, DynamoDB, or Azure equivalents).
• Event-Driven Architectures: Solid experience with messaging brokers and streaming platforms (Kafka, Solace, SQS) and asynchronous processing patterns.
• Engineering Fundamentals: Strong mastery of Microservices architecture, distributed systems design, and Test-Driven Development (TDD).
• Active AI Coding Experience: Practical, daily workflow integration of AI coding agents (Claude Code, Amazon Q, Copilot, or similar) for code creation, refactoring, and test suite generation.
• Database & SQL Fundamentals: Strong working knowledge of relational database schemas, query optimization, and data modeling.
• Legacy Integration (C# / .NET): Ability to read and understand C# / .NET code to analyze legacy business logic during migration (not required for active feature development).
• Financial Domain Knowledge: Prior experience working in financial risk, trading platforms, settlement systems, or handling financial data formats (fpML, XML).
• DevOps & Infrastructure: Experience with Infrastructure-as-Code (Terraform, CloudFormation) and container technologies (Docker, ECS, EKS).
• Frontend Exposure: Basic familiarity with React, Angular, or TypeScript (purely helpful for understanding legacy system UI flows).