Pyspark Developer

Tata Consultancy Services; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services in Bengaluru seeks a senior specialist with 5–8 years of experience in credit risk and regulatory risk domains to develop and tune ML-driven risk models using PySpark, Spark, Python, and Scala on Big Data platforms.

You will collaborate with risk analytics teams to enhance models, perform feature engineering, training, scoring, and production deployment, ensuring compliance with regulatory guidelines.

Qualifications

• Advanced analytics and statistical analysis using large data sets from bureau/vendor data.
• Experience developing and deploying risk models in credit/banking domains.
• Proficient in PySpark/Scala to validate and implement models and codes in risk contexts.

Responsibilities

• Collaborate with risk management teams on risk model requirements for consumer and wholesale banking.
• Enhance ML models using PySpark/Scala and productionize models from development to deployment.
• Lead feature engineering, model training, scoring, and retraining activities in data pipelines.
• Architect data pipelines and automate data ingestion and model jobs across environments.

Skills

PySpark

Spark

Python

Scala

Hive

MapReduce

Unix shell scripting

Credit risk domain

Big Data

Education

Master’s degree in Statistics/Mathematics/Finance/Engineering

Tools

Hadoop

Spark

MapReduce

Job description

• Functional Skills: Experience in Credit Risk/Regulatory risk domain
• Technical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scripting
• Good to Have Skills: Exposure to Machine Learning Techniques
• Experience range: 5-8 years
• Job Description:

5+ Years of experience with Developing/Fine tuning and implementing programs/applications

Using Python/PySpark/Scala on Big Data/Hadoop Platform.

Roles and Responsibilities:

• a) Work with a Leading Bank’s Risk Management team on specific projects/requirements pertaining to risk Models in consumer and wholesale banking
• b) Enhance Machine Learning Models using PySpark or Scala
• c) Work with Data Scientists to Build ML Models based on Business Requirements and Follow ML Cycle to Deploy them all the way to Production Environment
• d) Participate Feature Engineering, Training Models, Scoring and retraining
• e) Architect Data Pipeline and Automate Data Ingestion and Model Jobs

Skills and competencies:

Required:

• Strong analytical skills in conducting sophisticated statistical analysis using bureau/vendor data, customer performance Data and macro-economic data to solve business problems.
• Working experience in languages PySpark & Scala to develop code to validate and implement models and codes in Credit Risk/Banking
• Experience with distributed systems such as Hadoop/MapReduce, Spark, streaming data processing, cloud architecture.
• Familiarity with machine learning frameworks and libraries (like scikit-learn, SparkML, tensorflow, pytorch etc.
• Experience in systems integration, web services, batch processing
• Experience in migrating codes to PySpark/Scala is big Plus
• The ability to act as liaison conveying information needs of the business to IT and data constraints to the business applies equal conveyance regarding business strategy and IT strategy, business processes and workflow
• Flexibility in approach and thought process
• Attitude to learn and comprehend the periodical changes in the regulatory requirement as per FED

Education Qualification: Master’s degree with a specialization in Statistics, Mathematics, Finance or Engineering Degree