Senior PySpark Lead for Surveillance Data Platforms

Luxoft; Województwo pomorskie; Vor Ort; PLN 260.000 - 360.000; Vollzeit

Zusammenfassung

Luxoft seeks an experienced senior Python/Big Data engineer to join the Surveillance Enhancement program in Poland. You will build and optimize data pipelines using PySpark, implement scalable architectures, and lead development tasks in a fast-paced regulatory environment.

Responsibilities include requirements analysis, design, development, testing and deployment, with a focus on MAR/Market Abuse regulatory needs.

Qualifikationen

• 15+ years of experience in data intensive PySpark development, window functions, performance optimization, Spark architecture, salting and column pruning.
• Proven experience leading a development team.
• Proven experience as a core Python developer.

Aufgaben

• Develop software applications based on business requirements.
• Maintain software applications and implement enhancements per project specs.
• Participate in requirement analysis, design, development, testing and deployment activities.
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate functionality.
• Ensure tasks are completed within deadlines and adhere to coding standards.

Kenntnisse

Pyspark

Window Functions

Performance Optimization

Spark Architecture

Salting

Column Pruning

Lead Development Team

Core Python

Gen AI

OOPS/Exception Handling

Data Pipelines

Git

Data Modeling

Tools

Django

PostgreSQL

Apache Airflow

Jira

Jobbeschreibung

Luxoft seeks an experienced senior Python/Big Data engineer to join the Surveillance Enhancement program in Poland. You will build and optimize data pipelines using PySpark, implement scalable architectures, and lead development tasks in a fast-paced regulatory environment.

Responsibilities include requirements analysis, design, development, testing and deployment, with a focus on MAR/Market Abuse regulatory needs.