Staff Quality Engineer - AI

symplr®; Bengaluru; On-site; INR 3,000,000 - 5,000,000; Full time

Job summary

symplr® is seeking an experienced AI QE Engineer to validate cutting-edge AI solutions across generative, conversational, and predictive models in Bengaluru, India. You will drive quality for AI APIs and services hosted on AWS using Python-based test automation and data validation.

Collaborating with data scientists and developers, you will assess model performance, explainability, data quality, and production ML lifecycle validation, including deployment monitoring and regression testing.

Qualifications

• 9+ years of experience deploying and testing AI solutions, especially generative, conversational and predictive AI.
• Bachelor’s degree (B.E./B.Tech or equivalent) in a relevant field.
• Strong Python proficiency and hands-on experience with AI/ML libraries.
• Familiarity with LangChain and NLP frameworks for building conversational agents.

Responsibilities

• Test and validate AI/ML solutions including Generative AI, Conversational AI, and Predictive models.
• Understand end-to-end ML lifecycle from data preparation to deployment and monitoring.
• Evaluate model performance, data quality, and explainability outputs.
• Validate AI APIs and services built with FastAPI or Flask.
• Collaborate with data scientists, software engineers, and product teams.
• Perform regression, reliability, and scalability testing for AI systems.

Skills

Python

AI/ML Libraries

Model Validation

Data Quality

Pandas/NumPy

Explainability

FastAPI/Flask

AWS SageMaker

LangChain

Education

Bachelor's degree in related field

Tools

PyTorch

TensorFlow

Keras

scikit-learn

LangChain

Job description

Overview

We are seeking an experienced and talented AI QE Engineer to join our team. In this role, you will be responsible for validating cutting-edge artificial intelligence solutions across various domains, including generative AI, conversational AI, and predictive AI.

The application, hosted in AWS, includes EC2, S3, Lambda, Athena, DymanoDB, OpenSearch, CloudWatch, GLUE, Bedrock, SageMaker, Kendra, Amazon Q, Claude from Anthropic Titan Embeddings from in AWS, Python, Langchain, and Streamlit technologies.

Duties & Responsibilities

• Experience in testing and validating AI/ML solutions including Generative AI, Conversational AI, and Predictive models
• Strong understanding of how ML models are built end-to-end (data preparation, feature engineering, training, validation, tuning)
• Knowledge of core ML algorithms and model types (regression, classification, clustering, tree-based models, neural networks, transformers)
• Proficiency in Python for AI test automation, data analysis, and model output validation
• Hands-on experience with pandas, NumPy, and scikit-learn for data and model validation
• Experience in data quality analysis, profiling, and feature validation
• Understanding of model evaluation metrics and validation of performance results
• Ability to interpret model behavior and explainability outputs
• Experience testing AI APIs and services built using FastAPI or Flask
• Familiarity with cloud-based AI deployments, preferably AWS SageMaker
• Understanding of production ML lifecycle, including deployment validation and monitoring
• Strong analytical, problem-solving, and communication skills for cross-functional collaboration

Desirable:

• Hands-on experience testing Generative AI prompts, hallucinations, and response quality
• Familiarity with Responsible AI, bias, fairness, and safety validation
• Knowledge of MLOps pipelines and CI/CD validation for ML systems Experience with performance, latency, and scalability testing for AI services
• Exposure to adversarial testing and edge-case validation for AI models
• Experience testing AI systems in regulated or high-risk domains

Skills Required

• Bachelor’s degree (B.E.) from four-year college or university, or equivalent combination of education and experience.
• 9+ years of experience in deploying and testing AI solutions, particularly in the areas of generative AI, conversational AI, and predictive AI.
• Strong proficiency in Python and experience with AI/ML libraries such as PyTorch, NumPy, scikit-learn, TensorFlow, and Keras
• Familiarity with LangChain and other NLP frameworks for building conversational agents and language models