Lead Pyspark Developer - Chennai/ Bengaluru/ Hyderabad (10-20 yrs) - Hybrid

Luxoft India; Bengaluru; Híbrido; INR 3.500.000 - 5.500.000; Jornada completa

Descripción de la vacante

Luxoft India is seeking a lead Pyspark developer for Bengaluru/Chennai/Hyderabad, hybrid role with 3 days in office. 10-20 years of experience required, able to join immediately or within 30 days.

The role involves building data-intensive pipelines, optimizing performance, and guiding a development team in a bank surveillance program. Strong Python and DB skills are essential.

Formación

• 10+ years of experience in data intensive Pyspark development, window functions, performance optimization and Spark architecture.
• Must have experience in leading a development team.
• Proven experience as a Core Python developer.
• Strong knowledge of DB connectivity, data loading, transformation and calculation.
• Experience with apply/map functions and concurrency in data pipelines.

Responsabilidades

• Develop software applications based on business requirements.
• Maintain software and implement enhancements per project specs.
• Participate in requirement analysis, design, development, testing and deployment.
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate functionality.
• Ensure development tasks are completed within deadlines.
• Adhere to coding standards and best practices.
• Mentor junior developers when needed.
• Perform code reviews and recommend improvements.
• Review requirements to design reliable applications.
• Create coding documentation and technical specifications.
• Act as primary contact for development queries.
• Analyze and resolve development issues accurately.

Conocimientos

Pyspark Development

Spark Architecture

Performance Optimization

Salting

Column Pruning

Window Functions

Team Leadership

Core Python

DB Connectivity

Apply Map/Map Functions

Herramientas

Git

Descripción del empleo

Pl note we are looking for lead Pyspark developer with Performance Optimization and Spark Architecture experience for Bengaluru/ Chennai and Hyderabad location who are fine to work in hybrid mode and can join immediate or within 30 days availability

Experience - 10-20 yrs

Work Mode- Hybrid (3 days per week in office)

Notice period - Immediate to max 30 days

Skills - Pyspark , Performance Optimization and Spark Architecture

Project Description

Surveillance Enhancement Program (SEP) is a multi-year program (MYP) to build and enhance the surveillance platform to detect potential instances of market misconduct, market manipulation and market abuse. The Bank met its commitment to FCA (Financial Conduct Authority, FCA) to meet Market Abuse Regulation ("MAR") requirements for critical data sources in September 2017. Following that, subsequent projects were initiated to further enhance, expand and complement the current coverage of the automated surveillance platform

Responsibilities

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed.
• Perform code reviews and recommend improvements.
• Review business requirements and recommend changes to develop reliable applications.
• Develop coding documentation and other technical specifications for assigned projects.
• Act as primary contact for development queries and concerns.
• Analyze and resolve development issues accurately.

Mandatory Skills

• 10+ years of experience in data intensive Pyspark Development ,Window Functions , Performance Optimization ,Spark Architecture,Salting ,Column Pruning
• Must have experience in leading a Development Team
• Proven experience as a Core Python developer
• Strong knowledge of DB connectivity, data loading , transformation, calculation
• Application of apply map, apply, map functions
• Concurrency and error handling data pipeline batch of size [1-10 gb]
• Ability to understand business requirements and translate them into technical requirements
• Ability to design architecture of data pipeline for concurrent data processing
• Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services

Work mode- Hybrid (3 days per week in office)

Experience - 10-20 yrs

Notice period - Immediate to 30 days max availability