Senior PySpark Lead Developer - Hybrid (Poland)

Luxoft Poland; Kraków; Hybrid; PLN 180,000 - 320,000; Full time

Benefits offered by this job

Private Medical & Dental care & Life

Internal Mobility program

Job summary

Luxoft Poland is seeking a Senior Data Engineer to develop and enhance the surveillance platform for regulatory compliance. The role focuses on building data-intensive PySpark applications, leading a development team, and translating business requirements into scalable data pipelines.

Collaboration across locations and projects is expected as part of the hybrid work model. The ideal candidate will have strong PySpark, Python, Pandas/Numpy skills, and familiarity with Gen AI, along with

Qualifications

• 10+ years of experience in data intensive PySpark development, Window Functions, Performance Optimization, Spark architecture, Salting, Column Pruning.
• Must have experience in leading a Development Team.
• Proven experience as a Core Python developer.
• Good understanding of Gen AI.
• Strong knowledge of DB connectivity, data loading, transformation, calculation.
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations.
• Vector operations on Pandas dataframes /series.
• Application of apply map, apply, map functions.
• Concurrency and error handling data pipeline batch of size [1-10 gb].
• Ability to understand business requirements and translate them into technical requirements.
• Ability to design architecture of data pipeline for concurrent data processing.
• Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services.

Responsibilities

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed.
• Perform code reviews and recommend improvements.
• Review business requirements and recommend changes to develop reliable applications.
• Develop coding documentation and other technical specifications for assigned projects.
• Act as primary contact for development queries and concerns.
• Analyze and resolve development issues accurately.

Skills

PySpark

Team Leadership

Python

Gen AI

DB Connectivity

Pandas

NumPy

Data Wrangling

Spark Architecture

Apply/Map Functions

Concurrency & Error Handling

Business Requirements to Tech

Git

Tools

Django

PostgreSQL

Job description

Luxoft Poland is seeking a Senior Data Engineer to develop and enhance the surveillance platform for regulatory compliance. The role focuses on building data-intensive PySpark applications, leading a development team, and translating business requirements into scalable data pipelines.

Collaboration across locations and projects is expected as part of the hybrid work model. The ideal candidate will have strong PySpark, Python, Pandas/Numpy skills, and familiarity with Gen AI, along with