Senior PySpark Lead for Market Surveillance Platform

Luxoft

Singapore

On-site

SGD 180,000 - 240,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Luxoft in Singapore is seeking an experienced Senior Data Engineer to lead a data-intensive PySpark development team. You will design and implement scalable data pipelines, optimize performance, and collaborate with stakeholders to translate business needs into robust technical solutions.

The role requires strong Python, Pandas/Numpy, and DB skills, with exposure to Gen AI concepts and workflow orchestration tools. A leadership mindset and ability to mentor junior developers are essential.

Qualifications

  • Experience in data-intensive PySpark development and performance optimization.
  • Proven experience leading a development team.
  • Strong Python development background with OOP, error handling, and parallel processing.
  • Solid DB connectivity, data loading, transformation and calculations.
  • Extensive Pandas/Numpy dataframes experience and data wrangling.

Responsibilities

  • Develop software applications based on business requirements.
  • Maintain and enhance applications per project specs.
  • Participate in requirement analysis, design, testing, and implementation.
  • Propose new techniques and technologies for development.
  • Perform unit testing and UAT to evaluate functionality.
  • Ensure tasks are completed within deadlines.
  • Follow coding standards and best practices; mentor juniors.
  • Conduct code reviews and suggest improvements.
  • Translate business requirements into technical solutions.
  • Create coding documentation and technical specs.
  • Act as primary contact for development queries.
  • Analyze and resolve development issues.

Skills

Data-intensive PySpark
Team leadership
Core Python
Gen AI awareness
OOP & exception handling
DB connectivity
Pandas/Numpy
Data wrangling & aggregations
Git & versioning
Concurrency in pipelines

Tools

PySpark
Pandas
NumPy
Airflow
Git
Django
PostgreSQL
Jira

Job description

Luxoft in Singapore is seeking an experienced Senior Data Engineer to lead a data-intensive PySpark development team. You will design and implement scalable data pipelines, optimize performance, and collaborate with stakeholders to translate business needs into robust technical solutions.

The role requires strong Python, Pandas/Numpy, and DB skills, with exposure to Gen AI concepts and workflow orchestration tools. A leadership mindset and ability to mentor junior developers are essential.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

luxoft information technology (singapore) pte. ltd. • Singapore

On-site
SGD 180,000 - 240,000
Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

Luxoft • Singapore

On-site
SGD 120,000 - 180,000
Lead PySpark Engineer – Spark & AI-Assisted Debugging
Lead PySpark Engineer – Spark & AI-Assisted Debugging

OX Consultancy Pte. Ltd. • Singapore

On-site
SGD 90,000 - 140,000
Senior Java Spark Big Data Engineer – Singapore
Senior Java Spark Big Data Engineer – Singapore

LION & ELEPHANTS CONSULTANCY PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000
Senior Lead Python/PySpark Engineer – AI-Driven Data Pipelines
Senior Lead Python/PySpark Engineer – AI-Driven Data Pipelines

JP Morgan Chase • Singapore

On-site
SGD 150,000 - 220,000
Senior Data Engineer - PySpark, Snowflake
Senior Data Engineer - PySpark, Snowflake

Optimum Solutions Pte Ltd • Singapore

On-site
SGD 120,000 - 180,000
Senior Big Data Engineering Manager
Senior Big Data Engineering Manager

Unison Consulting • Singapore

On-site
SGD 89,000 - 112,000
Senior Data Engineer – AWS/Databricks/PySpark Architect
Senior Data Engineer – AWS/Databricks/PySpark Architect

ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000
Senior Data Engineer – AWS / Databricks / PySpark
Senior Data Engineer – AWS / Databricks / PySpark

Astek Group • Singapore

On-site
SGD 120,000 - 180,000
Senior Data Engineer – AWS / Databricks / PySpark
Senior Data Engineer – AWS / Databricks / PySpark

ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000