Senior Developer - Python and Spark

Citigroup Inc.; Pune District; On-site; INR 2,800,000 - 4,200,000; Full time

Benefits offered by this job

Hybrid work arrangement

Exposure to AI/data platform projects

Growth in AI and platform architecture

Job summary

Citi is seeking a Senior Data Engineer in Pune, India to design and build scalable data processing platforms and analytics at scale. You will architect ETL/ELT pipelines with Python and PySpark, integrating Generative AI and ML models to enhance data flows and platform intelligence.

You will collaborate with cross-functional teams, implement AI-first approaches, and advance CI/CD maturity while delivering reliable data solutions and value across Citi’s data ecosystem.

Qualifications

• Proficient in Python and Apache Spark in production data systems.
• Advanced SQL and data transformation skills.
• Experience building ETL/ELT pipelines at scale.
• Familiar with Pandas, NumPy, SciPy for data wrangling.
• DevOps experience with CI/CD and GitHub.
• Knowledge of Linux environments; cross-functional collaboration.
• Experience with ML/AI workflows and data platforms.

Responsibilities

• Design and build scalable ETL/ELT pipelines in Python and PySpark for data ingestion and transformation.
• Develop production-grade code with code reviews and mentorship.
• Integrate Generative AI and machine learning models into data workflows.
• Establish data quality standards across all pipelines.
• Adopt an AI-first engineering approach with Agentic AI workflows.
• Automate CI/CD pipelines for builds and deployments.
• Apply performance tuning to process large datasets efficiently.
• Collaborate with engineers and product owners to align technical direction with delivery goals.

Skills

Python

Apache Spark

Spark SQL

SQL

Pandas

NumPy

SciPy

CI/CD

GitHub

Linux

Team collaboration

Tools

PySpark

Jupyter Notebook

GitHub

Job description

Citi is looking for a Senior Data Engineer to design and build the next-generation data processing and analytics platform that sits at the core of our AI‑driven business innovation. In this role, you will architect scalable ETL/ELT pipelines using Python and PySpark, while also integrating Generative AI and Machine Learning models to push the boundaries of what our data systems can do. Working within a high‑performing engineering team, you will shape how data flows, performs, and delivers value across the organisation at scale.

Responsibilities

• Design and build highly scalable ETL/ELT pipelines in Python and PySpark to power reliable data ingestion, transformation, and integration at scale.
• Develop production‑grade code and elevate team capability through structured code reviews and hands‑on peer mentorship.
• Integrate Generative AI and Machine Learning models into data workflows, using AI development tools to deliver innovative solutions and resolve complex technical challenges.
• Establish and enforce data quality standards across all pipelines, ensuring accuracy, consistency, and reliability of data throughout its lifecycle.
• Implement an AI‑first engineering approach by exploring and embedding Agentic AI workflows to increase development velocity and platform intelligence.
• Automate CI/CD pipelines for builds and deployments, advancing DevOps maturity across the data platform.
• Apply performance tuning and optimisation techniques to process large volumes of structured and semi‑structured data efficiently and at speed.
• Collaborate directly with engineers and product owners to align technical direction with delivery goals and accelerate outcomes.

Required Qualifications & Skills

• Deep hands‑on expertise in Python and Apache Spark, including Spark Core, Spark SQL, and DataFrames/Datasets, applied to production data systems.
• Advanced SQL capability across complex query writing, data transformation logic, and query performance optimisation.
• Demonstrated experience designing, building, and maintaining ETL/ELT pipelines for large‑scale data ingestion and transformation.
• Proficiency with Python data processing libraries including Pandas, NumPy, and SciPy for data wrangling and preprocessing tasks.
• Practical experience with DevOps tools and CI/CD pipeline automation, with version control managed through GitHub.
• Solid working knowledge of Linux environments, with familiarity across Windows systems.
• Ability to collaborate across multidisciplinary teams to diagnose complex technical problems and deliver effective, lasting solutions.

Beneficial Skills & Qualifications

• Working knowledge of Machine Learning and Generative AI, with direct exposure to AI development tooling in an engineering context.
• Familiarity with Agentic AI workflows and their application to enhancing platform capabilities or development productivity.
• Experience using Jupyter Notebook for rapid prototyping and iterative development of data solutions.

What We Offer

At Citi, you will work on technically challenging problems that matter, contributing to a platform that shapes how a global financial institution processes and derives intelligence from data. This is a role where technical depth, curiosity, and engineering rigour are genuinely valued.

• Hybrid working model with 3 days in the office and 2 days working remotely, giving you flexibility and in‑person collaboration.
• Access to complex, large‑scale engineering challenges at the intersection of data and AI, keeping your technical skills at the forefront of the industry.
• Opportunities to grow into AI and platform architecture disciplines, supported by a team culture that invests in continuous technical development.
• Exposure to a globally connected engineering network, collaborating with specialists across technology, data, and product functions.
• A performance‑driven environment where your contributions to platform innovation directly influence business outcomes at scale.
• Wellbeing and work‑life balance support, alongside a competitive benefits package designed to recognise the value you bring.

Build data systems that power real decisions — apply now to join Citi’s next‑generation AI and data platform engineering team.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.