Staff AI Quality Engineer

symplr; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Sodexo Meal Coupon

Internet Reimbursement

Mobile Reimbursement

Gym Reimbursement

Health Insurance - Personal, Term Life

Transport Facility - Subsidized Rate

Job summary

symplr is seeking a Staff AI Quality Engineer to validate AI/ML solutions across generative, conversational, and predictive domains. The role emphasizes robust model validation in production-like environments and cross-functional collaboration.

You will work with AWS-based tools (SageMaker, Lambda, OpenSearch) and collaborate with data scientists to validate data pipelines, training, and deployment processes. This is a remote-first role with Bangalore office presence.

Qualifications

• Bachelor's degree or equivalent in CS/EE or related field.
• 9+ years in deploying and testing AI solutions (generative, conversational, predictive).
• Proficiency in Python and AI/ML libraries (PyTorch, NumPy, scikit-learn, TensorFlow, Keras).

Responsibilities

• Test and validate AI/ML solutions including Generative AI, Conversational AI, and Predictive models.
• Understand end-to-end ML pipelines: data prep, feature engineering, training, validation, tuning.
• Evaluate model performance metrics and explainability outputs.
• Interpret model behavior and explainability results for stakeholders.
• Test AI APIs/services built with FastAPI or Flask.
• Validate production ML lifecycle, deployment validation, and monitoring.

Skills

Strong analytical skills

Communication skills

Education

Bachelor's degree in a relevant field

Tools

Python

PyTorch

NumPy

scikit-learn

TensorFlow

Keras

LangChain

NLP frameworks

AWS SageMaker

OpenSearch

CloudWatch

DynamoDB

EC2

S3

Bedrock

Titan Embeddings

Lambda

Pandas

FastAPI

Flask

Job description

Staff AI Quality Engineer

Overview

We are seeking an experienced and talented AI QE Engineer to join our team. In this role, you will be responsible for validating cutting-edge artificial intelligence solutions across various domains, including generative AI, conversational AI, and predictive AI.

The application, hosted in AWS, includes EC2, S3, Lambda, Athena, DymanoDB, OpenSearch, CloudWatch, GLUE, Bedrock, SageMaker, Kendra, Amazon Q, Claude from Anthropic Titan Embeddings from in AWS, Python, Langchain, and Streamlit technologies.

Duties & Responsibilities

• Experience in testing and validating AI/ML solutions including Generative AI, Conversational AI, and Predictive models
• Strong understanding of how ML models are built end-to-end (data preparation, feature engineering, training, validation, tuning)
• Knowledge of core ML algorithms and model types (regression, classification, clustering, tree-based models, neural networks, transformers)
• Proficiency in Python for AI test automation, data analysis, and model output validation
• Hands-on experience with pandas, NumPy, and scikit-learn for data and model validation
• Experience in data quality analysis, profiling, and feature validation
• Understanding of model evaluation metrics and validation of performance results
• Ability to interpret model behavior and explainability outputs
• Experience testing AI APIs and services built using FastAPI or Flask
• Familiarity with cloud-based AI deployments, preferably AWS SageMaker
• Understanding of production ML lifecycle, including deployment validation and monitoring
• Strong analytical, problem-solving, and communication skills for cross-functional collaboration

Desirable:

• Hands-on experience testing Generative AI prompts, hallucinations, and response quality
• Familiarity with Responsible AI, bias, fairness, and safety validation
• Knowledge of MLOps pipelines and CI/CD validation for ML systems Experience with performance, latency, and scalability testing for AI services
• Exposure to adversarial testing and edge-case validation for AI models
• Experience testing AI systems in regulated or high-risk domains

Skills Required

• Bachelor’s degree (B.E.) from four-year college or university, or equivalent combination of education and experience.
• 9+ years of experience in deploying and testing AI solutions, particularly in the areas of generative AI, conversational AI, and predictive AI.
• Strong proficiency in Python and experience with AI/ML libraries such as PyTorch, NumPy, scikit-learn, TensorFlow, and Keras
• Familiarity with LangChain and other NLP frameworks for building conversational agents and language models

About symplr:

As a leader in healthcare operations solutions, we empower healthcare organizations to navigate the complexities of integrating critical business operations. Our customers are at the heart of everything we do, and they rely on our mission-critical systems to drive better operations and better outcomes.

We are a remote-first company with employees working across the United States, India, and the Netherlands. Guided by values, we focus on teamwork, championing our customers, being rooted in action and outcomes, overcoming challenges, and leading through equality and integrity. Read more about symplr's culture and values at symplr.com/careers.

Perks & Benefits:

• Sodexo Meal Coupon
• Internet Reimbursement
• Mobile Reimbursement
• Gym Reimbursement
• Health Insurance - Personal, Term Life Policy, Depends Cover, Dental Cover, OPD Cover & etc
• Transport Facility - Subsidized Rate

Working at symplr means working in the unlimited growth market of healthcare technology. Our multi-year contracts with clients bring recurring revenue that creates a stable financial situation. This enables us to provide employees a competitive salary, with additional benefits like insurance, lunch coupons, and reimbursement for mobile and broadband subscription and the gym.

symplr provides employees with opportunities to grow professionally through various activities including:

• Participation in new projects
• Internal weekly learning sessions with colleagues
• External learning opportunities
• Access to Udemy to grow your skillset
• Cross-departmental collaboration
• Activities with international colleagues
• Pursuit of OKRs specific to your role in the organization
• A buddy program for new employees

Your manager will be your coach, assisting you in maximizing your current skills and cultivating new ones as you grow on your career path at symplr. You'll have the opportunity to present your work achievements to the team and be awarded for milestone achievements.

You'll have regular time for fun and educational events, both digital and in-person at our Bangalore office. For example, feel free to show your pool table skills with your teammates, or take some time to relax in a massage chair.