Data Scientist - AI Evaluation and Responsible AI

NTT DATA Europe & Latam

Ro

In loco

EUR 90.000 - 130.000

Tempo pieno

39 ore fa
Candidati tra i primi

Una candidatura completa in un minuto

Curriculum e lettera di presentazione personalizzati, pronti da inviare.

Supera i filtri ATS

Descrizione del lavoro

NTT DATA Europe & Latam is seeking a Senior Data Scientist to own a shared evaluation framework, release gates, and responsible-AI controls across ML, RAG, agent, document and vision use cases. You’ll work directly with delivery teams from baseline creation through release decisions, production monitoring and quality reporting.

The role requires strong Python/SQL skills, production experience with AI/ML systems, and a deep grounding in statistics and experimental design.

Competenze

  • At least 10 years of professional experience preferred.
  • 6+ years of experience in data science, ML evaluation, model validation, experimentation or responsible AI.
  • 4+ years of strong Python and SQL experience with model, log and monitoring data.
  • Production experience evaluating and monitoring AI or ML systems.
  • Strong foundation in statistics, experimental design, error analysis and calibration.
  • Experience with RAG, LLM or agent evaluation and safety testing.
  • Strong technical communication, stakeholder influence, ownership, mentoring and judgment under ambiguity.
  • Excellent command of English, both written and spoken.

Mansioni

  • Define evaluation strategies, golden datasets, metrics, thresholds and release gates for ML, RAG, agent, document and vision solutions.
  • Build automated evaluation harnesses, regression tests, red-team and prompt-injection tests, and human-evaluation protocols.
  • Define responsible-AI controls covering explainability, bias, PII, human oversight, traceability and model risk.
  • Integrate evaluation into CI/CD and MLOps or LLMOps pipelines.
  • Specify monitoring, alerts, model cards and audit evidence, and investigate production quality issues.
  • Work hands-on with use-case teams from baseline creation to release decisions and report quality against business measures.
  • Challenge weak evidence or unsafe release decisions constructively.
  • Deliver reusable, tested Python code, evaluation assets and governance templates.

Conoscenze

Python
SQL
ML evaluation
Responsible AI
CI/CD
MLOps
Statistical analysis
RAG/LLM evaluation
Adversarial testing
Stakeholder communication

Strumenti

CloudWatch
New Relic
Langfuse
Bedrock Guardrails

Descrizione del lavoro

Who We Are

We operate beyond organizational silos, combining deep expertise in data engineering and AI-driven solutions to design and deliver scalable, user-centric applications and modern data architectures. We support a wide range of industries, including Automotive, Banking, Insurance, Telecommunications, E-commerce, and Public Services.

We operate beyond organizational silos, combining deep expertise in data engineering and AI-driven solutions to design and deliver scalable, user-centric applications and modern data architectures. We support a wide range of industries, including Automotive, Banking, Insurance, Telecommunications, E-commerce, and Public Services. We are looking for a Senior Data Scientist to own a shared evaluation framework, release gates and responsible-AI controls across machine learning, RAG, agent, document and vision use cases. The role works directly with delivery teams from baseline creation through release decisions, production monitoring and quality reporting.

What You’ll Be Doing
  • Define evaluation strategies, golden datasets, metrics, thresholds and release gates for ML, RAG, agent, document and vision solutions.
  • Build automated evaluation harnesses, regression tests, red-team and prompt-injection tests, and human-evaluation protocols.
  • Define responsible-AI controls covering explainability, bias, PII, human oversight, traceability and model risk.
  • Integrate evaluation into CI/CD and MLOps or LLMOps pipelines.
  • Specify monitoring, alerts, model cards and audit evidence, and investigate production quality issues.
  • Work hands-on with use-case teams from baseline creation to release decisions and report quality against business measures.
  • Challenge weak evidence or unsafe release decisions constructively.
  • Deliver reusable, tested Python code, evaluation assets and governance templates.
What You'll Bring Along
  • At least 10 years of professional experience preferred
  • 6+ years of experience in data science, ML evaluation, model validation, experimentation or responsible AI.
  • 4+ years of strong Python and SQL experience with model, log and monitoring data.
  • Production experience evaluating and monitoring AI or ML systems.
  • Strong foundation in statistics, experimental design, error analysis and calibration.
  • Experience with RAG, LLM or agent evaluation and safety testing.
  • Ability to challenge weak evidence or unsafe release decisions constructively.
  • Strong technical communication, stakeholder influence, ownership, mentoring and judgment under ambiguity.
  • Experience with adversarial testing, PII validation, Bedrock Guardrails, CloudWatch, New Relic, Langfuse or safety-sensitive applications is preferred.
  • Excellent command of English, both written and spoken
Ottieni la revisione del curriculum gratis e riservata.

o trascina qui il file.

Similar jobs

Offerte di lavoro simili che vale la pena confrontare

Senior Data Scientist: Responsible AI & ML Evaluation Lead
Senior Data Scientist: Responsible AI & ML Evaluation Lead

NTT DATA Europe & Latam • Ro

In loco
EUR 90.000 - 130.000
AI Solutions Architect
AI Solutions Architect

Msxinternational • Lombardia

In loco
EUR 90.000 - 140.000
AI Solutions Architect
AI Solutions Architect

Msxinternational • Roma

In loco
EUR 90.000 - 120.000
AI Engineer
AI Engineer

Expert.Ai S.P.A. • Modena

In loco
EUR 45.000 - 65.000
AI Engineer/Agentic Workflow Platform
AI Engineer/Agentic Workflow Platform

NTT DATA Europe & Latam • Emilia-Romagna

In loco
EUR 80.000 - 120.000
Senior Machine Learning Engineer
Senior Machine Learning Engineer

Skillvue • Milano

Remoto
EUR 70.000 - 90.000
Competitive compensation
Flexible work
Budget for conferences and training
+1
Data & Validation Manager
Data & Validation Manager

Cosmo I Building Health Confidence • Roma

In loco
EUR 75.000 - 95.000
Responsible Ai Engineer
Responsible Ai Engineer

Accenture PLC • Bardi

Ibrido
EUR 80.000 - 120.000
Senior Generative AI / AI Solution Architect
Senior Generative AI / AI Solution Architect

NTT DATA Europe & Latam • Ro

In loco
EUR 90.000 - 130.000
Senior Ai/Ml Engineer
Senior Ai/Ml Engineer

Team • Puglia

In loco
EUR 120.000 - 180.000