Gen AI Expert

NomiSo, Inc; Bengaluru; Sur place; INR 4 000 000 - 7 000 000; Plein temps

Résumé du poste

NomiSo, Inc. seeks an AI Expert with 8+ years of GenAI and QA/testing expertise to architect production-grade AI systems and design comprehensive testing strategies at scale.

You will own AI implementations and quality validation end-to-end, bridging AI systems engineering with rigorous testing. This IC role emphasizes practical, production-ready GenAI delivery.

Qualifications

• 4+ years building and shipping LLM systems in production.
• 5+ years of Rigorous QA/testing experience.
• Ability to design retrieval systems, inference pipelines, and model evaluation.
• Experience with test design, test plans, and test automation frameworks.

Responsabilités

• Build end-to-end GenAI systems—RAG pipelines, fine-tuning strategies, prompt optimization.
• Own deployment of AI features with reliability and scalability.
• Instrument systems with telemetry and run production experiments.
• Mentor engineers on AI best practices and scale GenAI adoption.

Connaissances

AI Architect

Testing Domain Expert

Independent Operator

Production-Focused Builder

Team Amplifier

Continuous Experimenter

Formation

Bachelor/Equivalent in CS or Math

Outils

OpenAI API

Anthropic

Hugging Face

Description du poste

We're seeking an AI Expert with 8+ years of combined Gen AI and QA/Testing with Test automation expertise. This is a specialized IC role for someone who bridges AI systems engineering with rigorous quality assurance. You'll architect production-grade AI systems (RAG pipelines, fine-tuning, prompt optimization, inference scaling) while simultaneously designing comprehensive testing strategies, test cases, test plans, test automation, and execution frameworks that ensure AI systems behave reliably at scale. You own both the AI implementation and the quality validation end-to-end.

This role is for someone with deep testing domain knowledge who has evolved into Gen AI, or vice versa, equally strong in both. You're not just implementing off-the-shelf models; you're designing retrieval systems, optimizing inference pipelines, evaluating model trade-offs, and building systems that reliably serve millions of transactions. You understand that GenAI is as much about solving the "why" (business impact) as the "how" (technical implementation)

What We’re Looking For (Who You Are)

• AI Systems Architect: 4+ years of experience, and building and shipping LLM systems in production. You understand model selection, RAG architecture, fine-tuning strategies, inference optimization, and cost management. You make pragmatic trade-offs between accuracy, latency, and cost.
• Testing Domain Expert: Rigorous QA/testing background atleast 5+ years. You're fluent in test case design, test plan creation, test automation frameworks, test execution strategies, and test reporting. You understand different testing types (unit, integration, end-to-end, performance, regression, smoke, sanity) and when to apply each. You can architect testing strategies from scratch.
• Independent Operator: You own AI features end-to-end—from problem definition through evaluation, deployment, and production monitoring. You unblock yourself and make good decisions with incomplete information.
• Production-Focused Builder: You've shipped LLM systems at scale. Comfortable with tokenization, context windows, vector database optimization, handling hallucinations and failures.
• Team Amplifier: You mentor engineers on AI best practices, participate in code reviews, and help scale GenAI adoption.
• Continuous Experimenter: You A/B test approaches, iterate based on production data, and stay current with rapid LLM evolution.

Technical Stack & Qualifications

Category

Requirement

GenAI Core

• LLM APIs (OpenAI, Anthropic, Hugging Face), model trade-offs and selection
• Prompt engineering, fine-tuning (PEFT/LoRA), inference optimization
• Cost and latency management, hallucination detection and mitigation
• Agentic patterns, multi-step reasoning, reliability under uncertainty

Testing Skills

• Test case design (positive, negative, edge cases, boundary conditions)
• Test planning and coverage analysis
• Test types: Unit, integration, end-to-end, performance, regression, smoke, sanity testing
• Test execution, CI/CD pipeline setup, test reporting and metrics dashboards
• Defect tracking and root cause analysis
• AI-specific testing: Evaluation frameworks (accuracy, F1, precision, recall), regression testing for models, hallucination testing, bias detection, A/B testing for AI features, mocking LLM APIs in testing, handling non-deterministic outputs through deterministic test design

Education

Degree in Computer Science, Mathematics, or equivalent professional experience. Practical portfolio or published work demonstrating GenAI/ML implementation is valued highly.

Responsibilities

• Build: Design and implement end-to-end GenAI systems—RAG pipelines, fine-tuning strategies, prompt optimization. Make pragmatic choices between model options, latency/cost trade-offs, and production constraints.
• Ship: Own deployment of AI features. Ensure reliability, scalability, and proper handling of edge cases (hallucinations, rate limits, failures).
• Measure: Instrument systems with telemetry. Run experiments comparing approaches. Iterate based on production data and user feedback.
• Collaborate: Mentor engineers on AI best practices. Participate in code reviews. Help the team adopt and scale GenAI capabilities