PySpark Big Data Engineer

Citigroup Inc.; Pune District; In loco; INR 1.200.000 - 1.800.000; Tempo pieno

Descrizione del lavoro

Citigroup Inc. in Pune is seeking an Applications Development Intermediate Programmer Analyst to contribute to designing and implementing new or revised application systems in coordination with the Technology team.

The role focuses on data processing, PySpark pipelines, API/backend development, and integration of data from multiple sources, with exposure to Hadoop, Spark, and DevOps tools.

Competenze

• At least 4+ years’ experience in Python, SQL and PySpark.
• Proficiency in distributed data processing and big data tools: Hadoop, HDFS, YARN, Spark, Hive.
• APIs and backend development with FastAPI or Flask.
• DevOps basics: Git, CI/CD, Linux/Unix commands.
• Design, develop, and maintain big data pipelines using PySpark.
• Experience integrating data from multiple sources.
• Experience with DevTools like OpenShift, TeamCity, uDeploy, Bitbucket, GitHub.
• Proactively contribute to stability of production systems and troubleshoot key components.

Conoscenze

Python

SQL

PySpark

Big data concepts

Troubleshooting

Learning agility

Git

CI/CD basics

Linux/Unix

OpenShift

TeamCity

uDeploy

Bitbucket

GitHub

Hadoop

HDFS

YARN

Spark

Hive

FastAPI

Flask

Formazione

Bachelor's degree in technology/engineering

Strumenti

Python

SQL

PySpark

Hadoop

HDFS

YARN

Spark

Hive

FastAPI

Flask

Git

CI/CD

Linux/Unix

OpenShift

TeamCity

uDeploy

Bitbucket

GitHub

Descrizione del lavoro

The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.

Qualification

• At least 4+ years’ experience in Python, SQL and PySpark.
• Proficiency in distributed data processing and big data tools and technologies: Hadoop, HDFS, YARN, Spark, Hive
• API's and backend development (FASTAPI / FLASK)
• Devops: GIT, CI/CD basics, Linux/Unix commands
• Design, develop, and maintain big data pipelines using PySpark
• Experience with integration of data from multiple data sources
• Experience in DevTools like openshift, teamcity, uDeploy, BitBucket, GitHub
• Proactively contribute to stability of overall production systems and troubleshoot key components and processes
• Keep track of latest technology trends and proactively learn new technologies driving practical and reference implementations

Nice to have

• Experience with NoSQL databases, such as ElasticSearch
• understand of LLMs, prompt engineering, Data qyery systems (NL to SQL), basic knowledge of vector databases (FAISS, pinecone etc)

Qualifications

• Bachelor’s degree (preferably in technology /engineering or related field).

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

Job Family Group

Technology

Job Family

Applications Development

Time Type

Full time

Most Relevant Skills

Please see the requirements listed above.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.