Senior Pyspark Developer

Luxoft; Bengaluru; On-site; INR 2,800,000 - 4,500,000; Full time

Job summary

Luxoft in Bengaluru is seeking a Senior PySpark Developer to work on trade surveillance data projects for a leading investment bank client. You will develop data-intensive PySpark solutions, implement Python-based components, and design robust data pipelines with RESTful interfaces.

You will contribute to architecture, unit testing, and code reviews, ensuring high-quality deliverables and timely delivery. Strong knowledge of Pandas/Numpy and DB connectivity is required.

Qualifications

• 8+ years of experience in data intensive Pyspark development.
• Experience as a core Python developer.
• Basic experience in Gen/Agentic AI
• Experience developing Classes, OOPS, exception handling, parallel processing .
• Strong knowledge of DB connectivity, data loading , transformation, calculation.
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations.
• Lambda Functions, Decorators.
• Vector operations on Pandas dataframes /series.
• Application of applymap, apply, map functions.
• Concurrency and error handling data pipeline batch of size [1-10 gb].
• Ability to understand business requirements and translate them into technical requirements.
• Ability to design architecture of data pipeline for concurrent data processing.
• Familiar with creating/designing RESTful services and APIs.
• Familiar with application unit tests.
• Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services.
• Debugging application.

Responsibilities

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed.
• Perform code reviews and recommend improvements.
• Review business requirements and recommend changes to develop reliable applications.
• Develop coding documentation and other technical specifications for assigned projects.
• Act as primary contact for development queries and concerns.
• Analyze and resolve development issues accurately.

Skills

Pyspark development

Python development

Gen AI

OOP concepts

DB connectivity

Pandas/Numpy

RESTful APIs

Git

Unit testing

Data pipelines

Parallel processing

Tools

Django

PostgreSQL

Airflow

Jira

Job description

Project description

We need a Senior Pyspark Developer to work for a leading investment bank client in Trade Surveillance Team.

Responsibilities

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed.
• Perform code reviews and recommend improvements.
• Review business requirements and recommend changes to develop reliable applications.
• Develop coding documentation and other technical specifications for assigned projects.
• Act as primary contact for development queries and concerns.
• Analyze and resolve development issues accurately.

SKILLS

Must have

• 8+ years of experience in data intensive Pyspark development.
• Experience as a core Python developer.
• Basic experience in Gen/Agentic AI
• Experience developing Classes, OOPS, exception handling, parallel processing .
• Strong knowledge of DB connectivity, data loading , transformation, calculation.
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations.
• Lambda Functions, Decorators.
• Vector operations on Pandas dataframes /series.
• Application of applymap, apply, map functions.
• Concurrency and error handling data pipeline batch of size [1-10 gb].
• Ability to understand business requirements and translate them into technical requirements.
• Ability to design architecture of data pipeline for concurrent data processing.
• Familiar with creating/designing RESTful services and APIs.
• Familiar with application unit tests.
• Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services.
• Debugging application.

Nice to have

• Knowledge of web backend technology - Django, Python, PostgreSQL.
• Apache Airflow
• Atlassian Jira
• Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
• Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus