Senior PySpark Lead - Data Platform & RegTech

Luxoft; Wrocław; On-site; PLN 300,000 - 550,000; Full time

Job summary

Luxoft is seeking a senior data engineer in Wroclaw to develop and enhance the surveillance platform using PySpark and Python, focusing on scalable data pipelines and performance optimization.

The role requires strong leadership of a development team, experience with Pandas/Numpy, and familiarity with Gen AI concepts. Collaboration with cross-functional teams and adherence to coding standards are essential.

Qualifications

• 10+ years of data-intensive PySpark development.
• Experience leading a development team.
• Proven Python developer with OO skills and strong debugging ability.

Responsibilities

• Develop software applications from business requirements.
• Maintain and enhance applications according to project specs.
• Participate in analysis, design, development, testing and deployment.

Skills

PySpark

Data Pipelines

Pandas/Numpy

Python Core

Team Leadership

Gen AI

Spark Architecture

Window Functions

Salting

Column Pruning

DB Connectivity

Git

Concurrency

Tools

Django

PostgreSQL

Apache Airflow

Job description

Luxoft is seeking a senior data engineer in Wroclaw to develop and enhance the surveillance platform using PySpark and Python, focusing on scalable data pipelines and performance optimization.

The role requires strong leadership of a development team, experience with Pandas/Numpy, and familiarity with Gen AI concepts. Collaboration with cross-functional teams and adherence to coding standards are essential.