Senior AI Platform Engineer: LLMs & RAG

LTS; United States; Remote; USD 140,000 - 210,000; Full time

Benefits offered by this job

Comprehensive benefits

Cutting-edge tools & platforms

Salary transparency

Job summary

LTS is seeking an AI Platform and Harness Engineer to build and maintain the infrastructure, tooling, and evaluation frameworks powering enterprise AI solutions. You’ll develop reusable AI harnesses and evaluation tools, enabling secure, scalable development of LLMs, AI agents, RAG, and Generative AI apps.

You’ll collaborate with AI architects, software engineers, and data scientists to ensure solutions are scalable, secure, and cost-effective, with observability, CI/CD, and cloud deployments

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, AI, or related field.
• 5+ years in software/platform engineering, backend, DevOps, or cloud engineering.
• 2+ years building or supporting Generative AI/LLM applications.
• Strong Python programming experience.
• Experience with cloud platforms (AWS, Azure, GCP).
• Experience deploying with Docker & Kubernetes; Git & CI/CD; IaC.

Responsibilities

• Design, build, and maintain enterprise AI platform capabilities for LLMs, agents, RAG, and Generative AI apps.
• Develop reusable AI harnesses to automate testing, evaluation, and QA.
• Build AI evaluation frameworks: model accuracy, retrieval quality, latency, cost.
• Implement observability: telemetry, tracing, logging, dashboards, metrics.
• Build LLMOps pipelines for deployment, versioning, evaluation, experimentation, rollback.
• Design automated workflows for prompt testing, retrieval evaluation, and performance benchmarking.
• Develop internal tools for prompt management, model experimentation, AI performance optimization.
• Build scalable backend services and APIs for AI platforms and enterprise integrations.
• Collaborate to integrate LLMs, RAG pipelines, vector DBs, and agentic workflows.
• Support deployment of AI services across AWS, Azure, or Google Cloud with containerized architectures.
• Implement CI/CD and infrastructure automation; apply governance and Responsible AI controls.
• Evaluate emerging AI frameworks and evaluation methodologies to improve productivity.
• Troubleshoot production AI issues and improve platform reliability and security.
• Document engineering standards, platform architecture, and operational best practices.

Skills

Python

LLMs

RAG

AI agents

Embeddings

Vector DBs

Prompt engineering

Back-end systems

Education

Bachelors in CS/SE/AI/Data Science

Tools

Docker

Kubernetes

Git

CI/CD

IaC

Job description

LTS is seeking an AI Platform and Harness Engineer to build and maintain the infrastructure, tooling, and evaluation frameworks powering enterprise AI solutions. You’ll develop reusable AI harnesses and evaluation tools, enabling secure, scalable development of LLMs, AI agents, RAG, and Generative AI apps.

You’ll collaborate with AI architects, software engineers, and data scientists to ensure solutions are scalable, secure, and cost-effective, with observability, CI/CD, and cloud deployments