Lead Pyspark Developer

Luxoft

Singapore

Vor Ort

SGD 180.000 - 240.000

Vollzeit

14 Tage+

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Luxoft in Singapore is seeking an experienced Senior Data Engineer to lead a data-intensive PySpark development team. You will design and implement scalable data pipelines, optimize performance, and collaborate with stakeholders to translate business needs into robust technical solutions.

The role requires strong Python, Pandas/Numpy, and DB skills, with exposure to Gen AI concepts and workflow orchestration tools. A leadership mindset and ability to mentor junior developers are essential.

Qualifikationen

  • Experience in data-intensive PySpark development and performance optimization.
  • Proven experience leading a development team.
  • Strong Python development background with OOP, error handling, and parallel processing.
  • Solid DB connectivity, data loading, transformation and calculations.
  • Extensive Pandas/Numpy dataframes experience and data wrangling.

Aufgaben

  • Develop software applications based on business requirements.
  • Maintain and enhance applications per project specs.
  • Participate in requirement analysis, design, testing, and implementation.
  • Propose new techniques and technologies for development.
  • Perform unit testing and UAT to evaluate functionality.
  • Ensure tasks are completed within deadlines.
  • Follow coding standards and best practices; mentor juniors.
  • Conduct code reviews and suggest improvements.
  • Translate business requirements into technical solutions.
  • Create coding documentation and technical specs.
  • Act as primary contact for development queries.
  • Analyze and resolve development issues.

Kenntnisse

Data-intensive PySpark
Team leadership
Core Python
Gen AI awareness
OOP & exception handling
DB connectivity
Pandas/Numpy
Data wrangling & aggregations
Git & versioning
Concurrency in pipelines

Tools

PySpark
Pandas
NumPy
Airflow
Git
Django
PostgreSQL
Jira

Jobbeschreibung

About the Role

Surveillance Enhancement Program (SEP) is a multi-year program (MYP) to build and enhance the surveillance platform to detect potential instances of market misconduct, market manipulation and market abuse. The Bank met its commitment to FCA (Financial Conduct Authority, FCA) to meet Market Abuse Regulation (MAR) requirements for critical data sources in September 2017. Following that, subsequent projects were initiated to further enhance, expand and complement the current coverage of the automated surveillance platform. This project will focus on Alerts enhancements, Strategic data sourcing from FM order and trade lakes, UI / workflow enhancements to meet regulatory obligations (i.e. Market Abuse Regulation, Dodd Frank). This also includes Control Room & Suitability Monitoring related data, UI and scenario enhancements.

Responsibilities
  • Develop software applications based on business requirements
  • Maintain software applications and make enhancements according to project specifications
  • Participate in requirement analysis, design, development, testing, and implementation activities
  • Propose new techniques and technologies for software development.
  • Perform unit testing and user acceptance testing to evaluate application functionality
  • Ensure to complete the assigned development tasks within the deadlines
  • Work in compliance with coding standards and best practices
  • Provide assistance to Junior Developers when needed.
  • Perform code reviews and recommend improvements.
  • Review business requirements and recommend changes to develop reliable applications.
  • Develop coding documentation and other technical specifications for assigned projects.
  • Act as primary contact for development queries and concerns.
  • Analyze and resolve development issues accurately.
Required Skills
  • 15+ years of experience in data intensive Pyspark development ,Window Functions , Performance Optimization ,Spark Architecture,Salting ,Column Pruning
  • Must have experience in leading a Development Team
  • Proven experience as a Core Python developer
  • Good understanding of Gen AI
  • Experience developing Classes, OOPS, exception handling, parallel processing
  • Strong knowledge of DB connectivity, data loading , transformation, calculation
  • Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
  • Lambda Functions, Decorators
  • Vector operations on Pandas dataframes /series
  • Application of apply map, apply, map functions
  • Concurrency and error handling data pipeline batch of size [1-10 gb]
  • Ability to understand business requirements and translate them into technical requirements
  • Ability to design architecture of data pipeline for concurrent data processing
  • Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services
  • Debugging application
Preferred Skills
  • Nice-to-Have Skills De ion:
  • Knowledge of web backend technology - Django, Python, PostgreSQL.
  • Apache Airflow
  • Atlassian Jira
  • Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
  • Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus
Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Lead Pyspark Developer
Lead Pyspark Developer

luxoft information technology (singapore) pte. ltd. • Singapore

Vor Ort
SGD 180.000 - 240.000
Lead Pyspark Developer
Lead Pyspark Developer

DXC Luxoft • Singapore

Vor Ort
SGD 120.000 - 180.000
Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

Luxoft • Singapore

Vor Ort
SGD 120.000 - 180.000
PySpark Lead
PySpark Lead

OX Consultancy Pte. Ltd. • Singapore

Vor Ort
SGD 90.000 - 140.000
Software Engineer (Apache / Pyspark / Python) - Contract Role
Software Engineer (Apache / Pyspark / Python) - Contract Role

QUESS ENGINEERING PTE. LTD. • Singapore

Vor Ort
SGD 120.000 - 180.000
Senior PySpark Lead for Market Surveillance Platform
Senior PySpark Lead for Market Surveillance Platform

luxoft information technology (singapore) pte. ltd. • Singapore

Vor Ort
SGD 180.000 - 240.000
Sr Lead Software Engineer - Python, Pyspark, Java
Sr Lead Software Engineer - Python, Pyspark, Java

JP Morgan Chase • Singapore

Vor Ort
SGD 180.000 - 240.000
Sr Lead Software Engineer - Python, Pyspark, Java
Sr Lead Software Engineer - Python, Pyspark, Java

Security Capital Research • Singapore

Vor Ort
SGD 180.000 - 240.000
Senior Data Engineer AWS / Databricks / PySpark
Senior Data Engineer AWS / Databricks / PySpark

astek singapore innovation technology pte. ltd. • Singapore

Vor Ort
SGD 120.000 - 190.000
Senior Lead Software Engineer, Java, Equities Prime - Securities Lending Technology
Senior Lead Software Engineer, Java, Equities Prime - Securities Lending Technology

JPMorganChase • Singapore

Vor Ort
SGD 180.000 - 240.000