Specialized Analytics Senior Analyst

Citigroup Inc.; Pune District; Presencial; INR 1.800.000 - 3.200.000; Jornada completa

Descripción de la vacante

Citigroup Inc. in Pune, India is hiring an AVP (C12) to lead analytical AI initiatives within the TTS Analytics team. You will own end‑to‑end delivery of AI‑powered products, build scalable evaluation pipelines, curate high‑quality data, and shape automated testing and deployment strategies.

You will apply 8+ years of data science/ML experience, with 5+ years in data science roles and 1–2 years in Generative AI, NLP, and evaluation frameworks. Strong Python and LangChain tooling are essential.

Formación

• Total 8+ years of experience in data science/ML with 5+ years in relevant roles.
• At least 1–2 years dedicated to Generative AI, LLMs, NLP and evaluation frameworks.
• Strong proficiency in Python and modern LLM tooling (LangChain, LlamaIndex, Hugging Face).
• Experience implementing AI governance, SDKs, and model risk management.

Responsabilidades

• Own end‑to‑end delivery of AI‑powered products: requirements, design, implementation, testing, deployment and support.
• Design, build and maintain automated evaluation pipelines and frameworks.
• Curate ground‑truth datasets covering diverse use cases and adversarial vectors.
• Develop prompt architectures and optimize token budgets for production workloads.
• Collaborate with platform, MLOps and data teams to integrate artifacts into CI/CD pipelines.
• Lead governance processes and author model risk documentation.

Conocimientos

Python (async/concurrency)

FastAPI/Pydantic

pytest

Pandas

NumPy

LangChain

LlamaIndex

Hugging Face

LLM tooling

AI governance

Educación

Bachelor/Master in CS/DS

Herramientas

S3

Descripción del empleo

Overview

The C12 is a developing professional role. Deals with most problems independently and has some latitude to solve complex problems. Integrates in-depth specialty area knowledge with a solid understanding of industry standards and practices. Good understanding of how the team and area integrate with others in accomplishing the objectives of the sub function/ job family. Applies analytical thinking and knowledge of data analysis tools and methodologies. Requires attention to detail when making judgments and recommendations based on the analysis of information. Typically deals with variable issues with potentially broader business impact. Applies professional judgment when interpreting data and results. Breaks down information in a systematic and communicable manner. Developed communication and diplomacy skills are required to exchange potentially complex/sensitive information. Moderate but direct impact through close contact with the businesses' core activities. Quality and timeliness of service provided will affect the effectiveness of own team and other closely related teams.

TTS Analytics C12

What do we do?

• The TTS Analytics team provides analytical insights to the Product, Pricing, Client Experience and Sales functions within the global Treasury & Trade Services business. The team works on business problems focused on enhancing client experience, driving acquisitions, cross-sell and revenue growth.


• The team extracts relevant insights, identifies business opportunities, converts business problems into analytical frameworks, uses big data tools and AI/ML techniques to drive data driven business outcomes in collaboration with business and product partners.


• The team works on building and operating Generative AI and deep learning solutions - Design, implement, and scale production‑grade AI applications end‑to‑end—from data ingestion and model services to user‑facing interfaces, observability, and secure deployments.

ROLE DESCRIPTION

The role will be AVP (C12) in the TTS Analytics team.

• As a key contributor to ideation on analytical projects to tackle strategic business priorities.


• The role will require endless curiosity, as ambiguity and open-ended questions are a core part of the team’s work.


• Own end‑to‑end delivery of AI‑powered products: requirements, design, implementation, testing, deployment, and support.


• Design, build, and maintain automated, scalable evaluation pipelines and frameworks (e.g., using LLM‑Eval‑Harness, or custom internal frameworks).


• Curate, version, and synthetically augment high‑quality ground‑truth ("golden") datasets covering diverse use cases, edge cases, domain‑specific tasks, and adversarial vectors.


• Implement metric suites evaluating task accuracy, semantic similarity, faithfulness, hallucination rates, toxicity, relevancy, bias, deterministic constraint satisfaction, and tool/agent execution fidelity.


• Establish statistically sound LLM-as-a-Judge evaluation protocols, human‑in‑the‑loop (HITL) calibration workflows, and deterministic/statistical baseline checks (BLEU, ROUGE, Exact Match, BERTScore).


• Formulate automated test suites and migration runbooks to transition production workloads across model versions with minimal disruption and validated parity.


• Develop structured, reproducible prompt architectures using advanced techniques (Few‑Shot, Chain‑of‑Thought, ReAct, DSPy, Automated Prompt Optimization).


• Analyze and optimize token budgets, prompt caching, model routing strategies (e.g., small SLMs for routing/extraction vs. large frontier models for complex reasoning), and speculative decoding strategies.


• Enforce schema compliance (JSON schema, Pydantic, function calling / tool definitions) and deterministic constraints across varying foundation model backends.


• Partner closely with platform architects, MLOps, backend engineers, and data engineers to integrate evaluation artifacts directly into CI/CD pipelines and deployment workflows.


• Lead the AI governance process


• Author comprehensive Object Development Documentation (ODD), Model Validation packages, model risk assessments, and technical artifacts detailing model design, limitation boundaries, training/evaluation data lineage, and testing methodologies.


• Engage early with new AI initiative owners to scope evaluation criteria, acceptable error thresholds, bias assessments, and testing requirements prior to production onboarding.


• Appropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.


• Ability to build partnerships with cross‑function leaders.

QUALIFICATIONS

Experience:

• Overall 8+ years of experience. 5+ years of Data Science/ML engineering experience, with at least 1–2 years dedicated to Generative AI, LLMs, NLP, and evaluation frameworks.

Must have substantial experience in:

• Strong proficiency in Python (async/concurrency, FastAPI/Pydantic, pytest) and standard data analysis stacks (Pandas, NumPy).


• Hands‑on experience with modern LLM tooling such as LangChain, LlamaIndex, Hugging Face, etc..


• Deep understanding of quantitative evaluation metrics (Semantic similarity, Faithfulness, Answer Relevancy, Precision/Recall, Hallucination scoring, Toxicity).


• Experience building structured dataset curation pipelines, data cleansing, versioning (like S3), and synthetic data generation techniques.


• Observability: application monitoring, centralized logging, tracing; strong debugging and problem‑resolution skills.


• AI Governance

Good to Have

• Prior experience in financial services, healthcare, or heavily regulated enterprise environments, particularly with Model Risk Governance


• Experience evaluating complex multi‑step reasoning, tool‑calling agents, memory systems, and Multi‑Agent Orchestration frameworks.


• Knowledge of data privacy and responsible AI practices; experience implementing audit and guardrail tooling.

Education:

• Bachelor / Masters (preferred) in Computer Science Engineering, Data Science, AI/ML, or related technical disciplines (or equivalent practical experience).

This job description provides a high‑level review of the types of work performed. Other job‑related duties may be assigned as required.

Job Family Group

Decision Management

Job Family

Specialized Analytics (Data Science/Computational Statistics)

Time Type

Full time

Most Relevant Skills

Please see the requirements listed above.

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.