Senior PySpark Lead for Market Surveillance Platform

Luxoft

Singapore

On-site

SGD 120,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Luxoft is seeking a senior data engineer to join a surveillance enhancement program. You will lead a data-intensive PySpark development team, design data pipelines, and optimize performance while aligning with MAR/Dodd-Frank regulations.

You will work on core Python development, integrate with PostgreSQL/other data sources, and apply Gen AI techniques to improve data processing and analytics. The role requires strong architecture skills and a proactive, collaborative approach.

Qualifications

  • 15+ years of experience in data-intensive PySpark development and performance optimization.
  • Experience leading a development team and delivering complex data pipelines.
  • Proven Python developer with strong OO design and testing practices.
  • Knowledge of Gen AI in data processing and analytics.
  • Strong DB connectivity and data loading/transformation skills.
  • Experience with Pandas/Numpy and large-scale data wrangling.

Responsibilities

  • Develop software applications based on business requirements.
  • Maintain and enhance software per project specs.
  • Participate in requirements analysis, design, development, testing, and implementation.
  • Review requirements and propose architecture for data pipelines.

Skills

PySpark
Lead a Development Team
Core Python
Gen AI
DB connectivity
Pandas/Numpy dataframes
Lambda Functions
Decorators
Vector operations
Concurrency

Tools

Git
PostgreSQL

Job description

Luxoft is seeking a senior data engineer to join a surveillance enhancement program. You will lead a data-intensive PySpark development team, design data pipelines, and optimize performance while aligning with MAR/Dodd-Frank regulations.

You will work on core Python development, integrate with PostgreSQL/other data sources, and apply Gen AI techniques to improve data processing and analytics. The role requires strong architecture skills and a proactive, collaborative approach.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

luxoft information technology (singapore) pte. ltd. • Singapore

On-site
SGD 180,000 - 240,000
Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

Luxoft • Singapore

On-site
SGD 180,000 - 240,000
Lead Pyspark Developer
Lead Pyspark Developer

luxoft information technology (singapore) pte. ltd. • Singapore

On-site
SGD 180,000 - 240,000
Lead Pyspark Developer
Lead Pyspark Developer

DXC Luxoft • Singapore

On-site
SGD 120,000 - 180,000
Lead Pyspark Developer
Lead Pyspark Developer

Luxoft • Singapore

On-site
SGD 180,000 - 240,000
Delivery Manager, Financial Markets & Trade Surveillance
Delivery Manager, Financial Markets & Trade Surveillance

Luxoft Holding • Singapore

On-site
SGD 180,000 - 240,000
Senior Lead Python/PySpark Engineer – AI-Driven Data Pipelines
Senior Lead Python/PySpark Engineer – AI-Driven Data Pipelines

JP Morgan Chase • Singapore

On-site
SGD 150,000 - 220,000
Senior Lead Python/PySpark Engineer for AI‑Driven Reg Reporting
Senior Lead Python/PySpark Engineer for AI‑Driven Reg Reporting

Aumni • Singapore

On-site
SGD 180,000 - 260,000
Senior Lead Python/PySpark/Java Engineer – Reg Reporting
Senior Lead Python/PySpark/Java Engineer – Reg Reporting

JP Morgan Chase • Singapore

On-site
SGD 180,000 - 240,000
Senior PL/SQL Developer - Banking Data & Reporting
Senior PL/SQL Developer - Banking Data & Reporting

Luxoft • Singapore

On-site
SGD 120,000 - 180,000