Full Stack AI Assurance Engineer

LTM; Pune District, Chennai District, Bengaluru; Hybrid; INR 2,500,000 - 3,800,000; Full time

Job summary

LTM is seeking a dynamic Full Stack AI Assurance Engineer in Pune to design and develop test cases and datasets for LLM evaluation using AI Trust Assure frameworks. You will supervise data quality, governance, and risk mitigation across the AI lifecycle.

Ideal candidates have 5–8 years in AIML and software QA, with hands-on coding using Copilot, Claude, and Crewai. You will collaborate with data scientists and compliance teams to strengthen AI reliability and safety.

Qualifications

• Bachelors or Masters in Computer Science, Data Science, Engineering or related field.
• 5-8 years of experience in AIML, data science, software quality assurance or related domain.
• Hands-on coding experience with AI tools (Copilot, Claude) and prompt engineering.
• Strong understanding of AIML concepts, model classification and lifecycle.
• Experience with risk management, testing, validation and monitoring of AI systems.
• Familiarity with data quality assessment, bias detection and fairness evaluation tools.

Responsibilities

• Identify and analyze data for model evaluation, addressing ethical, legal, safety and security risks.
• Evaluate data pipelines for quality, integrity and representativeness in AIML models.
• Develop and execute validation plans for AIML models covering performance and robustness.
• Implement tools to enhance interpretability and transparency of AI models.
• Assess AI systems for ethics and fairness and suggest mitigation strategies.
• Set up monitoring for AI models in production and address drifts or policy violations.
• Collaborate with data scientists, ML engineers and compliance teams to embed assurance practices.
• Contribute to AI governance frameworks and best practices; perform code reviews and automation improvements.

Skills

Python programming

Machine Learning

NLP

Prompt engineering

LLM evaluation

Data classification

Model governance

Code reviews

Education

Bachelor's or Master's in CS/DS/Engineering

Related field acceptable

Tools

Copilot

Claude

Crewai

Job description

• Top skills Python Programming Advance Machine Learning Libraries NLP Data
• Classification Model Classification Prompt Engineering API RAG LLM Evaluation
• Metrics RAGAS or any equivalent Lang Chain Hugging Face CICD About the Role
• We are seeking a dynamic Full Stack AI Assurance Engineer to design develop the test cases with data sets for LLM Evaluation using AI Trust Assure framework and test
• AILLM evaluations and recommendations The ideal candidate will have a strong foundation in AIML concepts Python programming and RAGAS any equivalent frameworks along with handson experience in prompt engineering and models classification data classification

Key Responsibilities

• Identify and analyse the Data preparations for Model Evaluation specific to AIML systems including ethical legal safety security and operational risks and Evaluate potential for bias unfairness lack of transparency and robustness etc
• Evaluate and monitor data pipelines to ensure high data quality integrity and representativeness for AIML models
• Develop and execute validation plans for AIML models including performance robustness fairness and security testing
• Implement tools and practices to enhance the interpretability and transparency of AI models and outcomes
• Assess AI systems for ethical and fairness considerations and implement or recommend mitigation strategies as needed
• Set up and manage monitoring systems for AI models in production identifying and addressing drifts anomalies or policy violations
• Work closely with data scientists ML engineers compliance teams and
• business stakeholders to embed assurance practices throughout the AI lifecycle
• Contribute to the development and continuous improvement of AI governance frameworks and best practices
• Perform code reviews debugging and continuous improvement of automation pipelines
• Stay updated on emerging trends in AI ML and DL for AI Assurance

Required Skills

Bachelors or Masters degree in Computer Science Data Science Engineering

or a related field

5-8 years of experience in AIML data science software quality assurance or a related domain

Hands on Exp in vibe coding such as Copilot Crewai Claude code

Strong understanding of AIML concepts model classification and lifecycle

Experience with risk management testing validation and monitoring of AI systems

Experience with data quality assessment bias detection and fairness evaluation tools

Familiarity with explainable AI XAI techniques and tools

Additional Skills

Experience with Prompt Engineering LLM APIs and chatbot integration

Knowledge of regulatory frameworks and standards for AI eg EU AI Act

ISOIEC standards

Knowledge of cloud environments AWS Azure or GCP

Excellent analytical problemsolving and documentation skills

Strong analytical debugging and communication skills

Skills

Mandatory Skills : AI/GenAI Research

Good to Have Skills : AI/ML Awareness Testing, AI/ML Testing