Senior AI/LLM Automation QA Engineer

Eleks; Buenos Aires; On-site; ARS 138,487,000 - 200,037,000; Full time

Benefits offered by this job

Close collaboration with a customer

Challenging tasks

Competence development

Influence on project technologies

Professional team

Dynamic environment

Job summary

ELEKS is seeking a Senior Automation Test Engineer (AI/LLM) in Argentina to join our QA team embedded with a US health-insurance client. You will help ensure quality of AI-powered applications, including LLM/RAG components and prompt safety.

Responsibilities include building Python/PyTest tests, REST API validation (FastAPI), data checks with SQL/Databricks, and CI/CD integration with Jenkins, Docker, and Git in a fast-paced, global delivery environment.

Qualifications

• Experience testing LLM/RAG behavior and prompt safety.
• Proficient in Python and PyTest for automation.
• Experience testing REST APIs and FastAPI-based apps.
• SQL and Databricks data validation experience.
• Experience with Azure AI Search and Azure OpenAI APIs.

Responsibilities

• Develop, execute, and maintain automated tests using Python and PyTest.
• Test REST APIs and FastAPI applications.
• Validate data with SQL and Databricks.
• Test Azure AI Search and Azure OpenAI APIs.
• Evaluate LLM intent routing and RAG retrieval quality.
• Perform prompt-injection and PII leakage testing.
• Validate groundedness, hallucinations, and citations in LLM responses.
• Conduct LLM evaluations using RAGAS/DeepEval.
• Automate regression testing with JSONL datasets.
• Use Postman/Newman for API testing; integrate with Jenkins CI/CD.
• Use Docker and Git in test automation tasks.
• Monitor applications with Application Insights/OpenTelemetry.

Skills

Python

PyTest

REST APIs

FastAPI

SQL

Databricks

Azure OpenAI

Azure AI Search

LLM/RAG testing

prompt safety

CI/CD

Docker

Git

Education

Bachelor's degree in a relevant field

Tools

Postman

Newman

Jenkins

OpenTelemetry

Application Insights

Job description

ELEKS is seeking a Senior Automation Test Engineer (AI/LLM) in Argentina to join our QA team embedded with a US health-insurance client. You will help ensure quality of AI-powered applications, including LLM/RAG components and prompt safety.

Responsibilities include building Python/PyTest tests, REST API validation (FastAPI), data checks with SQL/Databricks, and CI/CD integration with Jenkins, Docker, and Git in a fast-paced, global delivery environment.