Senior Analyst-GenAI

Naukri Assist; Hyderabad, Chennai District; Vor Ort; INR 1.200.000 - 2.400.000; Vollzeit

Zusammenfassung

Naukri Assist in Hyderabad, India seeks an experienced AI/ML Engineer to design and maintain data pipelines in Python, and collaborate with cross-functional teams to translate business needs into scalable solutions.

The role focuses on GenAI/LLM testing, retrieval augmented generation, and agent-based systems, with emphasis on evaluation, reliability, and automated data quality checks across cloud workflows and CI/CD pipelines.

Qualifikationen

• 5-8 years in software engineering, data science, or AI/ML.
• 3+ years focused on AI/LLM/GenAI testing or agent-based systems.
• Python scripting, automation, and debugging.
• Strong PySpark experience in distributed testing and data validation.
• GenAI concepts: LLMs, prompting, context management, RAG pipelines, tool-calling.
• Experience with agent development and deployment frameworks such as LangGraph, AutoGen, CrewAI, Copilot Studio Agent SDK, Vertex AI/OpenAI agent frameworks.
• Understanding of agent architecture: skills, tools, connectors, memory, guardrails, observability.
• Familiarity with GenAI/agent evaluation frameworks like Langsmith evaluation and AutoGen utilities.
• Testing foundation: functional, regression, scenario/edge-case, LLM safety & hallucination testing.
• Experience creating evaluation datasets and defining success criteria for AI behavior.
• API testing, data engineering concepts, cloud workflows (Azure/GCP/AWS), and CI/CD for test automation.
• Excellent communication for cross-functional collaboration.

Aufgaben

• Design, develop, and maintain complex data pipelines using Python to extract insights from large datasets.
• Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
• Develop and implement machine learning models using retrieval augmented generation techniques to improve data quality and accuracy.
• Troubleshoot issues with existing systems, identifying root causes and implementing fixes to ensure optimal performance.

Kenntnisse

Python

PySpark

GenAI concepts

LLM testing

Agent architecture

CI/CD testing

Software testing

Communication skills

Tools

LangGraph

AutoGen

CrewAI

Copilot Studio Agent SDK

Vertex AI

OpenAI agent frameworks

Jobbeschreibung

Roles and Responsibilities :

• Design, develop, and maintain complex data pipelines using Python to extract insights from large datasets.
• Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
• Develop and implement machine learning models using retrieval augmented generation techniques to improve data quality and accuracy.
• Troubleshoot issues with existing systems, identifying root causes and implementing fixes to ensure optimal performance.

Job Requirements :

• 5 - 8 years of overall experience in software engineering, data science, or AI/ML development, with at least 3+ years focused on AI/LLM/GenAI testing or agent-based systems.
• Python expertise in scripting, automation, and debugging.
• Strong PySpark experience in distributed testing, data validation, and pipeline testing.
• Hands-on knowledge of GenAI concepts, including LLMs, prompting, context management, RAG pipelines, agent tool-calling, and multi-agent orchestration.
• Experience with agent development and deployment frameworks such as LangGraph, AutoGen, CrewAI, Copilot Studio Agent SDK, and Vertex AI/OpenAI agent frameworks.

• Solid understanding of agent architecture covering skills, tools, connectors, memory, guardrails, and observability.

• Familiarity with modern GenAI/agent evaluation frameworks such as Langsmith evaluation, AutoGen agent-behavior assessment utilities etc. for benchmarking reliability, grounding, tool-use correctness, and multi-agent performance.
• Strong foundation in functional and regression testing, scenario and edge-case testing, LLM safety and hallucination testing, and workflow validation.
• Experience creating evaluation datasets and defining success criteria for AI behavior.
• Good understanding of API testing frameworks, data engineering concepts, cloud workflow execution (Azure/GCP/AWS), and CI/CD pipelines for test automation.

Excellent communication skills for cross-functional collaboration. .