Data Annotator - Applied AI Engineer Job ID: 427389

Masai School

India

Remote

INR 1,102,000 - 2,204,000

Part time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Masai School is seeking experienced Applied AI Engineers, AI Engineers, and Generative AI Engineers to contribute to data annotation, technical evaluation, and AI model training projects.

The role focuses on advanced evaluation of LLM-based apps, RAG pipelines, agent workflows, and AI-generated outputs, requiring deep understanding of modern AI architectures and practical implementation of AI-powered applications.

Qualifications

  • 2+ years of professional experience in Applied AI Engineering, AI Engineering, Generative AI, or related roles.
  • Strong hands-on experience with Python.
  • Practical experience building or maintaining LLM-powered applications.
  • Strong understanding of LLMs, transformer-based architectures, and generative AI concepts.
  • Experience with LLM APIs such as OpenAI, Anthropic, Gemini, or open-source models.
  • Practical experience with Retrieval-Augmented Generation (RAG), embeddings, and vector databases.
  • Familiarity with AI agent frameworks, tool calling, and workflow orchestration.
  • Experience with LangChain, LlamaIndex, or similar technologies.
  • Understanding of prompt engineering, context management, and LLM evaluation methodologies.
  • Familiarity with model fine-tuning, inference pipelines, and AI application deployment.
  • Ability to read, understand, and evaluate existing AI application codebases.
  • Experience with Git/GitHub and collaborative software development workflows.
  • Strong analytical, problem-solving, and technical reasoning abilities.
  • Excellent written communication skills to explain technical decisions and evaluation outcomes.

Responsibilities

  • AI Application Analysis: review and understand existing AI-powered applications and assess technical correctness.
  • LLM Architecture Evaluation: analyze transformer-based models and integration workflows.
  • RAG Pipeline Evaluation: assess embeddings, vector databases, and retrieval mechanisms.
  • AI Agent Evaluation: evaluate multi-agent workflows and orchestration logic.
  • Technical Data Annotation: annotate prompts, responses, code, and agent workflows.
  • AI-Generated Output Evaluation: assess factual accuracy and contextual relevance.
  • Prompt Engineering & Evaluation: review prompts and system instructions for quality.
  • Code-Level Assessment: review Python implementations and suggest corrections.
  • AI Model Integration & Fine-Tuning: evaluate APIs, integrations, and pipelines.
  • AI Framework Evaluation: assess LangChain, LlamaIndex usage.
  • Documentation & Reasoning: provide clear technical explanations and feedback.
  • Quality Assurance: maintain annotation accuracy and adhere to guidelines.

Skills

Python programming
LLM concepts
AI/ML architectures
Problem solving
Communication
Git/GitHub

Tools

Python
LangChain
LlamaIndex
OpenAI API
Vector databases
Git/GitHub

Job description

About the Role

We are looking for experienced Applied AI Engineers, AI Engineers, and Generative AI Engineers to contribute to specialised data annotation, technical evaluation, and AI model training projects. This is not a conventional data annotation role. The work involves advanced technical evaluation of LLM-based applications, AI agent workflows, retrieval-augmented generation (RAG) systems, prompt engineering, and AI-generated code and outputs. Selected professionals will work on tasks that require a strong understanding of modern AI architectures, generative AI frameworks, LLM integrations, agentic workflows, and practical implementation of AI-powered applications. The objective is to improve the accuracy, reasoning, performance, and reliability of AI systems by generating, reviewing, annotating, and evaluating technically complex datasets and AI-generated outputs.

Key Responsibilities
  • AI Application Analysis: Review and understand existing AI-powered applications, analyse implementation logic, evaluate system architecture, and assess technical correctness.
  • LLM Architecture Evaluation: Analyse large language models, transformer-based architectures, model configurations, and LLM integration workflows.
  • RAG Pipeline Evaluation: Assess retrieval-augmented generation systems, embedding models, vector databases, retrieval mechanisms, and context-generation workflows.
  • AI Agent Evaluation: Evaluate AI agent architectures, tool-calling mechanisms, multi-agent workflows, orchestration logic, and autonomous task execution.
  • Technical Data Annotation: Perform high-quality annotations on AI-related datasets, including prompts, responses, code, agent workflows, and technical documentation.
  • AI-Generated Output Evaluation: Assess AI-generated outputs for factual accuracy, logical reasoning, contextual relevance, instruction adherence, and technical correctness.
  • Prompt Engineering & Evaluation: Review and evaluate prompts, system instructions, prompt chains, and AI-generated responses for quality and effectiveness.
  • Code-Level Assessment: Review Python-based AI implementations, identify logical errors, evaluate integration approaches, and suggest technically sound corrections.
  • AI Model Integration & Fine-Tuning: Evaluate LLM APIs, model integrations, fine-tuning approaches, inference pipelines, and model deployment workflows.
  • AI Framework Evaluation: Assess implementations built using frameworks such as LangChain, LlamaIndex, and other AI application development frameworks.
  • Documentation & Reasoning: Provide clear technical explanations, annotations, justifications, and structured feedback for evaluation tasks.
  • Quality Assurance: Maintain high annotation accuracy and consistency while following project-specific guidelines and evaluation standards.
Required Skills & Qualifications
  • 2+ years of professional experience in Applied AI Engineering, AI Engineering, Generative AI, or related technical roles.
  • Strong hands-on experience with Python.
  • Practical experience building or maintaining LLM-powered applications.
  • Strong understanding of LLMs, transformer-based architectures, and generative AI concepts.
  • Experience working with LLM APIs such as OpenAI, Anthropic, Gemini, or open-source models.
  • Practical experience with Retrieval-Augmented Generation (RAG), embeddings, and vector databases.
  • Familiarity with AI agent frameworks, tool calling, and workflow orchestration.
  • Experience with frameworks such as LangChain, LlamaIndex, or similar technologies.
  • Understanding of prompt engineering, context management, and LLM evaluation methodologies.
  • Familiarity with model fine-tuning, inference pipelines, and AI application deployment.
  • Ability to independently read, understand, and evaluate existing AI application codebases.
  • Experience with Git/GitHub and collaborative software development workflows.
  • Strong analytical, problem-solving, and technical reasoning abilities.
  • Excellent written communication skills to explain technical decisions and evaluation outcomes.
Time Commitment
  • Minimum 2 hours of dedicated work per day.
  • Part-time engagement with flexible working hours, subject to project deadlines and task requirements.
  • Hourly-based payment for completed work, as per project-specific compensation terms.
  • Professionals must be available consistently and meet the expected quality and delivery standards.
Ideal Candidates

This opportunity is suitable for working Applied AI Engineers, Generative AI Engineers, LLM Engineers, AI Application Developers, and professionals with practical experience in building, integrating, evaluating, or maintaining AI-powered applications and systems.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Annotator - Applied AI Engineer
Data Annotator - Applied AI Engineer

Jobgether • India

Remote
INR 3,895,000 - 10,387,000
Competitive hourly compensation (USD 计
Fully remote in India
Flexible part-time engagement (min 2 h
Applied AI / LLM Specialist (Non-Developer)
Applied AI / LLM Specialist (Non-Developer)

Marvin • Delhi

On-site
INR 1,200,000 - 2,000,000
Associate AI Data Engineer
Associate AI Data Engineer

EXL • Maharashtra

On-site
INR 1,500,000 - 2,500,000
AI Data Engineer
AI Data Engineer

EXL • Maharashtra

On-site
INR 3,500,000 - 6,000,000
AI Engineers
AI Engineers

Aiden AI • Telangana

On-site
INR 2,500,000 - 4,500,000
Cutting-edge AI
Enterprise architecture
Mentorship
+1
Generative AI Engineer
Generative AI Engineer

TAC Security • Delhi

On-site
INR 1,800,000 - 2,500,000
AI Data Engineer
AI Data Engineer

ExlService Holdings, Inc. • Gurgaon

On-site
INR 1,500,000 - 2,100,000
AI NLP Engineer
AI NLP Engineer

Evalueserve India • Gurugram District

On-site
INR 800,000 - 1,500,000
AI Software Architect (LLM & Agentic Systems)
AI Software Architect (LLM & Agentic Systems)

Aziro • Hyderabad

Hybrid
INR 3,000,000 - 6,000,000
Artificial Intelligence Engineer
Artificial Intelligence Engineer

S&A Law Offices • Delhi

On-site
INR 1,200,000 - 2,000,000