Specialized Analytics Senior Analyst

Citi; Pune District; On-site; INR 3,800,000 - 7,000,000; Full time

Job summary

Citi is seeking an AVP (C12) in the TTS Analytics team in Pune to lead analytical projects and deliver AI-powered products. The role emphasizes end‑to‑end development, evaluation, and governance of AI solutions, leveraging LangChain, LlamaIndex, and modern ML tooling.

You will work with cross‑functional partners to drive data‑driven outcomes in treasury services. The position requires 8+ years of experience in data science/ML engineering, with strong Python, data curation, and governance skills.

Qualifications

• Strong Python expertise and modern ML tooling.
• Experience with data curation pipelines and dataset versioning.
• Familiarity with AI governance and evaluation frameworks.

Responsibilities

• Own end‑to‑end delivery of AI‑powered products: requirements, design, implementation, testing, deployment, and support.
• Design, build, and maintain automated evaluation pipelines and frameworks.
• Curate, version, and augment high‑quality ground‑truth datasets.
• Implement evaluation metrics for semantic similarity, faithfulness, and bias.
• Collaborate with platform, MLOps, and data engineering teams to CI/CD workflows.

Skills

Python

LangChain

LlamaIndex

HuggingFace

Evaluation metrics

Data curation

Data versioning

Observability

AI Governance

Education

Bachelor / Masters in CS/DS/AI

Tools

FastAPI

Pydantic

pytest

S3

Job description

Job Description

The C12 is a developing professional role. Deals with most problems independently and has some latitude to solve complex problems. Integrates in-depth specialty area knowledge with a solid understanding of industry standards and practices. Good understanding of how the team and area integrate with others in accomplishing the objectives of the sub function/ job family. Applies analytical thinking and knowledge of data analysis tools and methodologies. Requires attention to detail when making judgments and recommendations based on the analysis of information. Typically deals with variable issues with potentially broader business impact. Applies professional judgment when interpreting data and results. Breaks down information in a systematic and communicable manner. Developed communication and diplomacy skills are required to exchange potentially complex/sensitive information. Moderate but direct impact through close contact with the businesses' core activities. Quality and timeliness of service provided will affect the effectiveness of own team and other closely related teams.

Overview

The C12 is a developing professional role. Deals with most problems independently and has some latitude to solve complex problems. Integrates in-depth specialty area knowledge with a solid understanding of industry standards and practices. Good understanding of how the team and area integrate with others in accomplishing the objectives of the sub function/ job family. Applies analytical thinking and knowledge of data analysis tools and methodologies. Requires attention to detail when making judgments and recommendations based on the analysis of information. Typically deals with variable issues with potentially broader business impact. Applies professional judgment when interpreting data and results. Breaks down information in a systematic and communicable manner. Developed communication and diplomacy skills are required to exchange potentially complex/sensitive information. Moderate but direct impact through close contact with the businesses' core activities. Quality and timeliness of service provided will affect the effectiveness of own team and other closely related teams.

TTS Analytics C12

What do we do?

• The TTS Analytics team provides analytical insights to the Product, Pricing, Client Experience and Sales functions within the global Treasury & Trade Services business. The team works on business problems focused on enhancing client experience, driving acquisitions, cross-sell and revenue growth.
• The team extracts relevant insights, identifies business opportunities, converts business problems into analytical frameworks, uses big data tools and AI/ML techniques to drive data driven business outcomes in collaboration with business and product partners.
• The team works on building and operating Generative AI and deep learning solutions - Design, implement, and scale production‑grade AI applications end‑to‑end—from data ingestion and model services to user‑facing interfaces, observability, and secure deployments.

Role Description

The role will be AVP (C12) in the TTS Analytics team.

• As a key contributor to ideation on analytical projects to tackle strategic business priorities.
• The role will require endless curiosity, as ambiguity and open-ended questions are a core part of the team’s work.
• Own end‑to‑end delivery of AI‑powered products: requirements, design, implementation, testing, deployment, and support.
• Design, build, and maintain automated, scalable evaluation pipelines and frameworks (e.g., using LLM-Eval-Harness, or custom internal frameworks).
• Curate, version, and synthetically augment high‑quality ground‑truth ("golden") datasets covering diverse use cases, edge cases, domain‑specific tasks, and adversarial vectors.
• Implement metric suites evaluating task accuracy, semantic similarity, faithfulness, hallucination rates, toxicity, relevancy, bias, deterministic constraint satisfaction, and tool/agent execution fidelity.
• Establish statistically sound LLM-as-a-Judge evaluation protocols, human‑in‑the‑loop (HITL) calibration workflows, and deterministic/statistical baseline checks (BLEU, ROUGE, Exact Match, BERTScore).
• Formulate automated test suites and migration runbooks to transition production workloads across model versions with minimal disruption and validated parity.
• Develop structured, reproducible prompt architectures using advanced techniques (Few‑Shot, Chain‑of‑Thought, ReAct, DSPy, Automated Prompt Optimization).
• Analyze and optimize token budgets, prompt caching, model routing strategies (e.g., small SLMs for routing/extraction vs. large frontier models for complex reasoning), and speculative decoding strategies.
• Enforce schema compliance (JSON schema, Pydantic, function calling / tool definitions) and deterministic constraints across varying foundation model backends.
• Partner closely with platform architects, MLOps, backend engineers, and data engineers to integrate evaluation artifacts directly into CI/CD pipelines and deployment workflows.
• Lead the AI governance process
• Author comprehensive Object Development Documentation (ODD), Model Validation packages, model risk assessments, and technical artifacts detailing model design, limitation boundaries, training/evaluation data lineage, and testing methodologies.
• Engage early with new AI initiative owners to scope evaluation criteria, acceptable error thresholds, bias assessments, and testing requirements prior to production onboarding.
• Appropriately assess risk when business decisions are made, demonstrating consideration for the firm’s reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
• Ability to build partnerships with cross‑function leaders.

Qualifications

Experience:

• Overall 8+ years of experience. 5+ years of Data Science/ML engineering experience, with at least 1–2 years dedicated to Generative AI, LLMs, NLP, and evaluation frameworks.

Must have substantial experience in:

• Strong proficiency in Python (async/concurrency, FastAPI/Pydantic, pytest) and standard data analysis stacks (Pandas, NumPy).
• Hands‑on experience with modern LLM tooling such as LangChain, LlamaIndex, Hugging Face, etc..
• Deep understanding of quantitative evaluation metrics (Semantic similarity, Faithfulness, Answer Relevancy, Precision/Recall, Hallucination scoring, Toxicity).
• Experience building structured dataset curation pipelines, data cleansing, versioning (like S3), and synthetic data generation techniques.
• Observability: application monitoring, centralized logging, tracing; strong debugging and problem‑resolution skills.
• AI Governance

Good to Have

• Prior experience in financial services, healthcare, or heavily regulated enterprise environments, particularly with Model Risk Governance
• Experience evaluating complex multi‑step reasoning, tool‑calling agents, memory systems, and Multi‑Agent Orchestration frameworks.
• Knowledge of data privacy and responsible AI practices; experience implementing audit and guardrail tooling.

Education:

• Bachelor / Masters (preferred) in Computer Science Engineering, Data Science, AI/ML, or related technical disciplines (or equivalent practical experience).

This job description provides a high‑level review of the types of work performed. Other job‑related duties may be assigned as required.

Job Family Group:

Decision Management

Job Family:

Specialized Analytics (Data Science/Computational Statistics)

Time Type:

Full time

Most Relevant Skills

Please see the requirements listed above.

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.