Senior Software Engineer

Luxoft; Województwo mazowieckie; On-site; PLN 180,000 - 260,000; Full time

Benefits offered by this job

Laptop

Monitor

Job summary

Luxoft in Poland is seeking an experienced Senior PySpark Developer to drive data-intensive solutions. You will lead a development team, design scalable data pipelines, and implement performance optimizations using PySpark, Python, and Pandas/Numpy, aligning with regulatory and industry demands.

You will collaborate across engineering to deliver robust, scalable software, participate in design reviews, and mentor junior developers while ensuring best practices and timely delivery.

Qualifications

• 10+ years of data intensive PySpark development and related techniques.
• Experience leading a development team.
• Proven core Python development experience.
• Strong understanding of Gen AI and data pipelines.
• Deep knowledge of DB connectivity, data loading and transformation.
• Extensive Pandas/Numpy dataframe operations and data wrangling.

Responsibilities

• Develop software applications based on business requirements.
• Maintain applications and implement enhancements per specs.
• Participate in requirement analysis, design, development, testing, and deployment.
• Propose new techniques and technologies for software development.
• Perform unit testing and UAT to verify functionality.
• Complete development tasks within deadlines.
• Adhere to coding standards and best practices.
• Assist junior developers when needed.
• Conduct code reviews and suggest improvements.
• Review requirements and translate to technical solutions.
• Create coding documentation and technical specs.
• Act as primary contact for development queries.
• Analyze and resolve development issues.

Skills

PySpark

Window Functions

Performance Optimization

Spark Architecture

Salting

Column Pruning

Lead Development Team

Core Python

Gen AI

DB Connectivity

Data Loading

Data Transformation

Calculation

Pandas/Numpy

Slicing

Data Wrangling

Aggregations

Vector Operations

Apply/Map Functions

Concurrency

Error Handling

Git

Tools

Git

Job description

The Surveillance Enhancement Program (SEP) is a multi-year initiative aimed at developing and improving the surveillance platform to identify potential market misconduct, manipulation, and abuse. Our client fulfilled its commitment to the Financial Conduct Authority (FCA) by meeting Market Abuse Regulation (MAR) requirements for critical data sources in September 2017.

Subsequent projects have been launched to further enhance, expand, and complement the existing automated surveillance platform. This project will focus on alert enhancements, strategic data sourcing from FM order and trade lakes, and user interface and workflow improvements to comply with regulatory obligations such as Market Abuse Regulation and Dodd-Frank.

Additionally, the scope includes enhancements related to Control Room and Suitability Monitoring data, interfaces, and scenarios.

Responsibilities:

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed.
• Perform code reviews and recommend improvements.
• Review business requirements and recommend changes to develop reliable applications.
• Develop coding documentation and other technical specifications for assigned projects.
• Act as primary contact for development queries and concerns.
• Analyze and resolve development issues accurately.

Mandatory Skills Description:

• 10+ years of experience in data intensive Pyspark development ,Window Functions , Performance Optimization ,Spark Architecture,Salting ,Column Pruning
• Must have experience in leading a Development Team
• Proven experience as a Core Python developer
• Good understanding of Gen AI
• Strong knowledge of DB connectivity, data loading , transformation, calculation
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
• Vector operations on Pandas dataframes /series
• Application of apply map, apply, map functions
• Concurrency and error handling data pipeline batch of size [1-10 gb]
• Ability to understand business requirements and translate them into technical requirements
• Ability to design architecture of data pipeline for concurrent data processing
• Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services

What do we offer our employees

• Equipment, such as a laptop and monitor