Python Pyspark Developer

Luxoft India; Bengaluru; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Luxoft India is seeking a Senior PySpark Developer to join a leading investment bank's Trade Surveillance Team. The role focuses on data-intensive PySpark development, Python backend work, and building scalable data pipelines for financial data processing.

Responsibilities cover end-to-end SDLC, code reviews, and collaboration with junior developers while leveraging Django, PostgreSQL, and Jira in an agile setup. Strong emphasis on performance and accuracy in a surveillance domain.

Qualifications

• 8+ years of data intensive PySpark development experience required.
• Strong Python development background essential.
• Familiarity with Gen/Agentic AI is a plus.
• Solid knowledge of database connectivity, data loading, transformation and calculations.
• Extensive Pandas/Numpy experience for data wrangling and aggregations.
• Experience with Lambda functions and Decorators.
• Knowledge of vector operations on Pandas Series/DataFrames.
• Experience applying map/apply/applymap functions on dataframes.
• Capability to handle concurrent data pipelines (1-10 GB batches).
• Ability to translate business requirements into technical designs.
• Experience designing scalable data pipeline architectures for concurrency.
• Experience creating RESTful services and APIs.

Responsibilities

• Develop software applications based on business requirements.
• Maintain and enhance applications per project specs.
• Participate in end-to-end SDLC including testing and deployment.
• Propose new techniques and technologies for development.
• Perform unit and user acceptance testing to verify functionality.
• Ensure timely completion of tasks and adherence to coding standards.
• Provide guidance to junior developers as needed.
• Review code and suggest improvements.
• Analyze requirements and propose changes for reliable solutions.
• Create documentation and technical specs for projects.
• Serve as primary contact for development queries.
• Diagnose and resolve development issues accurately.

Skills

Pyspark

Python

Gen/Agentic AI

DB Connectivity

Pandas/Numpy

Lambda/Decorators

Vector Operations

DataFrame Functions

Concurrency

Requirements Analysis

Data Pipeline Architecture

REST APIs

Tools

Django

PostgreSQL

Jira

Job description

We need a Senior Pyspark Developer to work for a leading investment bank client in Trade Surveillance Team.

Responsibilities

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed.
• Perform code reviews and recommend improvements.
• Review business requirements and recommend changes to develop reliable applications.
• Develop coding documentation and other technical specifications for assigned projects.
• Act as primary contact for development queries and concerns.
• Analyze and resolve development issues accurately.

Mandatory Skills Description

• 8+ years of experience in data intensive Pyspark development.
• Experience as a core Python developer.
• Basic experience in Gen/Agentic AI
• Strong knowledge of DB connectivity, data loading , transformation, calculation.
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations.
• Lambda Functions, Decorators.
• Vector operations on Pandas dataframes /series.
• Application of applymap, apply, map functions.
• Concurrency and error handling data pipeline batch of size [1-10 gb].
• Ability to understand business requirements and translate them into technical requirements.
• Ability to design architecture of data pipeline for concurrent data processing.
• Familiar with creating/designing RESTful services and APIs.

  Nice-to-Have Skills Description

  • Knowledge of web backend technology - Django, Python, PostgreSQL.
  • Atlassian Jira
  • Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
  • Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus