Senior Automation Test Engineer (Ai/Llm)Argentina, Buenos Airesquality Assurance, Automation.

Eleks

Buenos Aires

On-site

ARS 138,487,000 - 200,037,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Close collaboration with a customer
Challenging tasks
Competence development
Influence on project technologies
Professional team
Dynamic environment

Job summary

ELEKS is seeking a Senior Automation Test Engineer (AI/LLM) in Argentina to join our QA team embedded with a US health-insurance client. You will help ensure quality of AI-powered applications, including LLM/RAG components and prompt safety.

Responsibilities include building Python/PyTest tests, REST API validation (FastAPI), data checks with SQL/Databricks, and CI/CD integration with Jenkins, Docker, and Git in a fast-paced, global delivery environment.

Qualifications

  • Experience testing LLM/RAG behavior and prompt safety.
  • Proficient in Python and PyTest for automation.
  • Experience testing REST APIs and FastAPI-based apps.
  • SQL and Databricks data validation experience.
  • Experience with Azure AI Search and Azure OpenAI APIs.

Responsibilities

  • Develop, execute, and maintain automated tests using Python and PyTest.
  • Test REST APIs and FastAPI applications.
  • Validate data with SQL and Databricks.
  • Test Azure AI Search and Azure OpenAI APIs.
  • Evaluate LLM intent routing and RAG retrieval quality.
  • Perform prompt-injection and PII leakage testing.
  • Validate groundedness, hallucinations, and citations in LLM responses.
  • Conduct LLM evaluations using RAGAS/DeepEval.
  • Automate regression testing with JSONL datasets.
  • Use Postman/Newman for API testing; integrate with Jenkins CI/CD.
  • Use Docker and Git in test automation tasks.
  • Monitor applications with Application Insights/OpenTelemetry.

Skills

Python
PyTest
REST APIs
FastAPI
SQL
Databricks
Azure OpenAI
Azure AI Search
LLM/RAG testing
prompt safety
CI/CD
Docker
Git

Education

Bachelor's degree in a relevant field

Tools

Postman
Newman
Jenkins
OpenTelemetry
Application Insights

Job description

ELEKS Quality Assurance Office is looking for a Senior Automation Test Engineer (AI/LLM) in Argentina

ABOUT PROJECT

Our customer is among the top 5 health insurance companies in the US. They are dedicated to expanding access to high-quality, cost-effective healthcare.

We are looking for an experienced Automation Test Engineer to join the project engagement and work embedded within the client’s delivery team, focusing on testing AI-powered applications, including Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) solutions.

REQUIREMENTS
  • Experience testing LLM/RAG behavior, including intent routing, retrieval quality, prompt safety, and evaluation/regression testing
  • Experience in test automation using Python and PyTest
  • Experience testing REST APIs and FastAPI-based applications
  • Experience with SQL and Databricks data validation
  • Experience testing Azure AI Search
  • Experience testing Azure OpenAI APIs
RESPONSIBILITIES
  • Develop, execute, and maintain automated tests using Python and PyTest
  • Perform REST API and FastAPI testing
  • Validate data using SQL and Databricks
  • Test Azure AI Search and Azure OpenAI APIs
  • Validate LLM intent routing and RAG retrieval quality
  • Perform prompt-injection and PII-leakage testing
  • Validate groundedness, hallucinations, and citations in LLM-generated responses
  • Conduct LLM evaluations using RAGAS or DeepEval
  • Perform automated regression testing using JSONL datasets
  • Use Postman and Newman for API testing
  • Integrate automated tests with Jenkins CI/CD
  • Use Docker and Git in test automation activities
  • Monitor applications using Application Insights or OpenTelemetry
WHAT YOU WILL GET WITH ELEKS
  • Close cooperation with a customer
  • Challenging tasks
  • Competence development
  • Ability to influence project technologies
  • Team of professionals
  • Dynamic environment with low level of bureaucracy
ABOUT

ELEKS is a custom software development company. We deliver value to our clients, thanks to our expertise and experience gained from working as a software innovation partner since 1991.

Our 2000+ professionals located in the Delivery Centers across Eastern Europe and sales offices in Europe and North America, provide our clients with a full range of software engineering services. These include product development, QA, R&D, design, technology consulting and dedicated teams.

Benefits

undefined

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Automation Test Engineer (AI/LLM)
Senior Automation Test Engineer (AI/LLM)

Eleks • Argentina

On-site
ARS 1,200,000 - 2,100,000
Senior Automation Test EngineerArgentina, Buenos AiresQuality Assurance, Automation.
Senior Automation Test EngineerArgentina, Buenos AiresQuality Assurance, Automation.

Eleks • Buenos Aires

On-site
ARS 1,200,000 - 2,400,000
Senior Automation Test Engineer – AI/LLM, PyTest & CI/CD
Senior Automation Test Engineer – AI/LLM, PyTest & CI/CD

Eleks • Argentina

On-site
ARS 1,200,000 - 2,100,000
Senior AI/LLM Automation QA Engineer
Senior AI/LLM Automation QA Engineer

Eleks • Buenos Aires

On-site
ARS 138,487,000 - 200,037,000
Close collaboration with a customer
Challenging tasks
Competence development
+3
Senior Automation Test Engineer — AI-Powered QA & CI/CD
Senior Automation Test Engineer — AI-Powered QA & CI/CD

Eleks • Buenos Aires

On-site
ARS 1,200,000 - 2,400,000
AI Quality Engineer
AI Quality Engineer

drvn • Rosario

On-site
ARS 1,800,000 - 3,000,000
Remote Qa Automation Engineer – Ai Testing
Remote Qa Automation Engineer – Ai Testing

Devconnectplatform • Buenos Aires

Hybrid
ARS 92,724,000 - 115,905,000
Professional growth
Competitive compensation
Exciting projects
+1
QA Automation Lead (Playwright / AI) IRC307024
QA Automation Lead (Playwright / AI) IRC307024

GlobalLogic Inc. • Argentina

On-site
ARS 1,800,000 - 3,000,000
Exciting Projects
Collaborative Environment
Work-Life Balance
+2
AI/LLM QA Engineer Lead – Automation & Gherkin
AI/LLM QA Engineer Lead – Automation & Gherkin

Eleks • Buenos Aires

On-site
ARS 38,434,000 - 76,867,000
Remote Qa Automation Engineer – Ci/Cd & Ai Testing
Remote Qa Automation Engineer – Ci/Cd & Ai Testing

Acciona IT • Buenos Aires

Hybrid
ARS 108,178,000 - 169,994,000
Professional growth
Competitive compensation
Exciting projects
+1