Senior PySpark Developer

Luxoft; Wrocław; On-site; PLN 180,000 - 260,000; Full time

Job summary

Luxoft is seeking a senior backend developer to join a Regulatory Reporting Platform for a large US investment bank. The role focuses on building scalable data pipelines using Python and PySpark, validating data quality, and delivering compliant regulatory outputs across many jurisdictions.

You will work with teams across Ukraine, Poland, and the US in an Agile setup, improving performance and reliability of high-volume data processing and contributing to reusable engineering components.

Qualifications

• 6+ years of commercial experience with Python.
• 2+ years of PySpark experience.
• Experience in designing and developing scalable, high-performance backend applications.
• Good knowledge of SQL and working with large datasets from multiple data sources.
• Strong problem-solving skills and ability to work effectively in a global distributed team.

Responsibilities

• Design, develop, and maintain solutions for a large-scale Regulatory Reporting Platform.
• Build and optimize data processing pipelines using Java and Apache Spark.
• Implement scalable data ingestion, transformation, and enrichment from multiple internal sources.
• Develop and optimize batch processing jobs for regulatory reports across jurisdictions.
• Ensure data quality, validation, reconciliation, and consistency throughout the reporting lifecycle.
• Collaborate with analysts, QA, and cross-functional teams to deliver reporting functionality.
• Troubleshoot issues in development, test, and production environments.
• Improve performance, scalability, and reliability for high-volume data processing.
• Contribute to reusable components and engineering best practices.
• Participate in code reviews, testing, and release activities.
• Work with distributed teams in Ukraine, Poland, and the US in an Agile setup.

Skills

Python

PySpark

Backend development

SQL

Distributed team collaboration

Tools

Spark

Job description

Project description

You will join a Regulatory Reporting Platform within a large US investment bank. The platform is responsible for collecting, processing, and transforming large volumes of data from multiple internal systems to produce regulatory reports for 240 jurisdictions worldwide. In total, the platform supports around 10,000 reports across a wide range of regulatory, compliance, financial, and risk reporting areas.This is a large-scale, business-critical data platform operating in a highly regulated environment, where data accuracy, quality, traceability, and timely delivery are essential. The team works on building and enhancing data pipelines, implementing reporting logic, ensuring data validation and reconciliation, and supporting the generation of complex regulatory outputs.The project is delivered by a global distributed team with colleagues located in Ukraine, Poland, and the US, offering close collaboration with engineers, analysts, and business stakeholders across multiple regions.

Responsibilities

• - Design, develop, and maintain solutions for a large-scale Regulatory Reporting Platform
• - Build and enhance data processing pipelines using Java and Apache Spark
• - Implement scalable logic for data ingestion, transformation, aggregation, and enrichment from multiple internal data sources
• - Develop and optimize batch processing jobs to support the generation of regulatory reports across multiple jurisdictions
• - Ensure high standards of data quality, validation, reconciliation, and consistency throughout the reporting lifecycle
• - Translate business and regulatory requirements into robust technical solutions
• - Collaborate with business analysts, QA engineers, and cross-functional teams to deliver reporting functionality
• - Investigate and resolve data and application issues in development, test, and production environments
• - Improve application performance, scalability, and reliability for high-volume data processing workloads
• - Contribute to the design and implementation of reusable components and engineering best practices
• - Participate in code reviews, testing, and release activities to ensure high-quality deliverables
• - Work closely with distributed teams across Ukraine, Poland, and the US in an agile delivery model
• - Support continuous improvement of the platform, including monitoring, troubleshooting, and operational stability

SKILLS

Must have

• - 6+ years of commercial experience with Python
• - 2+ years of PySpark experience
• - Experience in designing and developing scalable, high-performance backend applications
• - Good knowledge of SQL and working with large datasets from multiple data sources
• - Strong problem-solving skills and ability to work effectively in a global distributed team

Nice to have

• - Experience in financial services, investment banking, or regulatory reporting projects
• - Familiarity with data quality, reconciliation, and control frameworks
• - Experience working with large-scale enterprise data platforms
• - Knowledge of cloud environments and modern data platform architectures
• - Understanding of CI/CD practices and automated deployment pipelines
• - Experience working in Agile / Scrum teams within a distributed global setup