Senior AI Automation Engineer — AI Testing & Orchestration

Luxoft; Mexico; On-site; PHP 6,756,756 - 8,599,508; Full time

Job summary

Luxoft is seeking a Senior AI Automation Engineer to lead quality for an enterprise-level AI platform. You will build frameworks to validate Agentic AI workflows and LLM-based decision systems, and you’ll tackle challenging testing problems in AI with Playwright and modern CI/CD stacks.

The role emphasizes hands-on work across AWS Bedrock, vector databases, and multi-tool orchestration, shaping robust QA pipelines and scalable automation for cutting-edge AI initiatives.

Qualifications

• AWS Bedrock — hands-on: model access, Knowledge Bases, Lambda integration
• AI agents & Agentic tooling — practical knowledge of designing and operating AI agents, including agentic workflows, reusable skills, rules/guardrails, commands, and multi-tool/multi-agent orchestration
• RAG pipeline — end-to-end implementation: chunking, embedding, vector indexing, retrieval, generation
• Prompt engineering — zero-shot, few-shot, chain-of-thought, structured output (JSON mode), multi-turn
• Vector databases — working knowledge of OpenSearch, Pinecone, or Faiss; understands vector vs. graph DB difference
• Fine-tuning vs. RAG — ability to reason through which approach fits a given problem
• LLM orchestration — LangChain, LangGraph, or LlamaIndex
• Embeddings — understands semantic similarity; experience with Amazon Titan Embed or equivalent
• Python — for Lambda functions, AI pipeline scripting, and data processing
• Java — 3+ years of hands-on test automation development
• ReportPortal or equivalent test reporting tool
• REST API — concepts and hands-on usage
• Jenkins / CI-CD — pipeline debugging and integration
• AWS — S3, Lambda, API Gateway, IAM, OpenSearch Serverless
• Docker — containerized test execution environment

Skills

AI testing strategy

RAG pipeline knowledge

LLM orchestration

Embeddings understanding

Python scripting

Java development

Tools

AWS Bedrock

AI agents tooling

Vector databases

OpenSearch/Pinecone/Faiss

LangChain/LangGraph

Titan Embed

REST API

Jenkins / CI-CD

AWS services

Docker

OpenSearch Serverless

ReportPortal

Job description

Luxoft is seeking a Senior AI Automation Engineer to lead quality for an enterprise-level AI platform. You will build frameworks to validate Agentic AI workflows and LLM-based decision systems, and you’ll tackle challenging testing problems in AI with Playwright and modern CI/CD stacks.

The role emphasizes hands-on work across AWS Bedrock, vector databases, and multi-tool orchestration, shaping robust QA pipelines and scalable automation for cutting-edge AI initiatives.