Senior PySpark Lead for Market Surveillance Platform

luxoft information technology (singapore) pte. ltd.

Singapore

On-site

SGD 180,000 - 240,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

luxoft information technology (singapore) pte. ltd. seeks an experienced data engineering lead to build and enhance a scalable surveillance data platform. You will own PySpark-based pipelines, optimize performance, and guide a development team while coordinating with backend and data teams.

The role requires deep Python, Spark architecture knowledge, and strong SQL/dataframe experience to deliver compliant, high-quality software in a regulatory environment.

Qualifications

  • 15+ years of experience in data-intensive PySpark development and performance optimization.
  • Proven leadership of a development team and core Python expertise.
  • Strong understanding of data pipelines, transformations, and parallel processing.
  • Experience with Pandas/Numpy dataframes and SQL data loading.
  • Knowledge of distributed computing concepts and Spark architecture.

Responsibilities

  • Develop software applications from business requirements and maintain them.
  • Participate in analysis, design, development, testing, and deployment activities.
  • Lead code reviews and propose reliable, scalable solutions.
  • Translate business needs into technical architecture for data pipelines.
  • Collaborate with teams, ensure coding standards and best practices.

Skills

PySpark
Data pipelines
Spark architecture
Python core
Git
Airflow
PostgreSQL
Pandas/Numpy
Django/web backend
MAR regulatory knowledge

Tools

Git
Airflow
Jira

Job description

luxoft information technology (singapore) pte. ltd. seeks an experienced data engineering lead to build and enhance a scalable surveillance data platform. You will own PySpark-based pipelines, optimize performance, and guide a development team while coordinating with backend and data teams.

The role requires deep Python, Spark architecture knowledge, and strong SQL/dataframe experience to deliver compliant, high-quality software in a regulatory environment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

Luxoft • Singapore

On-site
SGD 180,000 - 240,000
Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

Luxoft • Singapore

On-site
SGD 120,000 - 180,000
Lead Pyspark Developer
Lead Pyspark Developer

luxoft information technology (singapore) pte. ltd. • Singapore

On-site
SGD 180,000 - 240,000
Lead Pyspark Developer
Lead Pyspark Developer

DXC Luxoft • Singapore

On-site
SGD 120,000 - 180,000
Lead Pyspark Developer
Lead Pyspark Developer

Luxoft • Singapore

On-site
SGD 180,000 - 240,000
Delivery Manager, Financial Markets & Trade Surveillance
Delivery Manager, Financial Markets & Trade Surveillance

Luxoft Holding • Singapore

On-site
SGD 180,000 - 240,000
Senior Lead Python/PySpark/Java Engineer – Reg Reporting
Senior Lead Python/PySpark/Java Engineer – Reg Reporting

JP Morgan Chase • Singapore

On-site
SGD 180,000 - 240,000
Senior Lead Python/PySpark Engineer – AI-Driven Data Pipelines
Senior Lead Python/PySpark Engineer – AI-Driven Data Pipelines

JP Morgan Chase • Singapore

On-site
SGD 150,000 - 220,000
Senior Lead Software Engineer - Python/PySpark/Java
Senior Lead Software Engineer - Python/PySpark/Java

JP Morgan Chase • Singapore

On-site
SGD 180,000 - 240,000
Senior Big Data Engineering Manager
Senior Big Data Engineering Manager

Unison Consulting • Singapore

On-site
SGD 89,000 - 112,000