AI Engineer: Agents & Evaluation

WOW Softech

Gurugram District

Sur place

INR 1 400 000 - 2 100 000

Plein temps

Il y a 5 jours
Soyez parmi les premiers à postuler

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Résumé du poste

WOW Softech is seeking a seasoned AI/ML Software Engineer to design and deploy production-grade LLM-powered applications. You will own agent design, retrieval and context layers, and evaluation harnesses in a fast-paced startup environment.

You will work with prompt architecture, schema-validated outputs, and cost-aware prompt design, ensuring robust operation and measurable improvements. Strong Python, Claude API experience, and production-grade tooling are essential.

Qualifications

  • 5+ years software engineering experience, including 2.5+ years building LLM-powered applications in production.
  • Demonstrable experience with retrieval-augmented generation and context engineering in a production system.
  • Experience building or operating systematic evaluation for LLM outputs (golden datasets, regression harnesses, rubric scoring).
  • Experience in a startup or small-team environment is a strong plus.
  • Bachelor's degree in Computer Science, Engineering, or related field, or equivalent demonstrable experience.

Responsabilités

  • Agent engineering: design and build BA, Dev, and QA agents with prompt architecture and schema-validated outputs.
  • Retrieval & context: build codebase retrieval layer and context-pack system.
  • Evaluation harness: own golden dataset and CI-driven nightly harness.
  • Failure modes: design for rejection loops, guard against hallucinations, and robust gap declarations.
  • Confidence & quality: implement a confidence scoring model for routing to senior review.
  • Token economics: manage per-invocation token cost tracking and model-mix tuning.
  • AI-native practice: daily work with agentic coding tools and product input.

Connaissances

Python proficiency
LLM integration
JSON-schema validation
Prompt engineering
Context engineering
Embeddings / vector search
Production experience
LangGraph
SQL fluency

Formation

Bachelor's in CS or related field
Equivalent demonstrable experience

Outils

pgvector
Claude API
Git
CI/CD
SQL

Description du poste

Duties & Accountabilities
  • Agent engineering: design and build the initial BA, Dev, and QA agents: prompt architecture, context assembly, tool use, and schema-validated JSON outputs, with prompts and skills packs versioned in git and treated as core IP.
  • Retrieval & context: build the codebase retrieval layer (syntax-aware chunking with tree-sitter, embeddings in pgvector) and the context-pack system: project glossary, org conventions, constraint register, and exemplar retrieval.
  • Evaluation harness: own the golden dataset (2530 expert-verified items) and the harness that runs nightly in CI: per-agent scores against published acceptance bars, trend monitoring, and regression alerts that block release.
  • Failure modes: design for the ways agents fail: schema-repair loops, reject-loop re-invocation with reviewer comments injected, retrieval-verified symbols to guard against hallucinated APIs, and honest gap declaration over coverage inflation.
  • Confidence & quality: implement and tune the confidence-scoring model that routes low-confidence outputs to senior review, and use every human modification (deltas captured at the gates) as improvement signal.
  • Token economics: own cost per story: per-invocation token and cost telemetry, model-mix tuning (fast tier for enrichment and test generation, top tier for code), and budget-aware prompt design.
  • AI-native practice: work daily with agentic coding tools; the friction you hit is product input.
Education & Experience
  • 5+ years of software engineering experience, including 2.5+ years building LLM-powered applications in real production use, not prototypes or side projects.
  • Demonstrable experience with retrieval-augmented generation and context engineering in a production system.
  • Experience building or operating systematic evaluation for LLM outputs (golden datasets, regression harnesses, rubric scoring).
  • Experience in a startup or small-team environment is a strong plus.
  • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent demonstrable experience.
Technical Skills Required
  • Strong Python and production experience with LLM provider APIs (Anthropic Claude family experience valued); structured outputs and JSON-schema validation.
  • Prompt and context engineering with measurable iteration, exemplar selection, context budgeting, instruction design for consistency.
  • Embeddings and vector search in production (pgvector or similar); code-aware chunking and indexing is a strong plus.
  • Evaluation engineering: building harnesses, defining acceptance bars, wiring them into CI, and reading the trends.
  • Token and cost optimisation: metering, model-tier routing, and cost-per-outcome thinking.
  • Agent orchestration frameworks (LangGraph or similar) are a strong plus, as is SQL fluency for telemetry analysis.
  • Practical, hands-on fluency with modern agentic coding tools (e.g., Claude Code or equivalent) is required.
Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Agentic/AI engineers with Claude/code/LLM skills1
Agentic/AI engineers with Claude/code/LLM skills1

ExlService Holdings, Inc. • Dadri

Sur place
INR 3 000 000 - 7 000 000
Engineer - Agentic AI
Engineer - Agentic AI

TIH | IIT Bombay • Mumbai

Sur place
INR 1 800 000 - 3 400 000
Agentic/AI engineers with Claude/code/LLM skills1
Agentic/AI engineers with Claude/code/LLM skills1

Mployee.me • Dadri

À distance
INR 1 800 000 - 3 200 000
Senior Manager - AI Engineering
Senior Manager - AI Engineering

PepsiCo Inc. • Hyderabad

Sur place
INR 1 500 000 - 2 100 000
Senior AI Quality Engineer
Senior AI Quality Engineer

Dataart Technologies • Bengaluru

Sur place
INR 1 200 000 - 2 400 000
Senior AI Data Engineer
Senior AI Data Engineer

ExlService Holdings, Inc. • Gurugram District

Sur place
INR 1 800 000 - 3 200 000
Agentic AI Developer
Agentic AI Developer

Vlink • Mumbai

Sur place
INR 1 200 000 - 2 400 000
Senior Artificial Intelligence Engineer
Senior Artificial Intelligence Engineer

Gaude Business And Infrastructure Solutions • Thiruvananthapuram

Sur place
INR 2 000 000 - 4 000 000
Senior AI Data Engineer
Senior AI Data Engineer

EXL • Gurugram District

Sur place
INR 2 500 000 - 4 500 000
Senior AI Data Engineer
Senior AI Data Engineer

EXL • Maharashtra

Sur place
INR 1 200 000 - 1 800 000