Senior LLMOps Engineer

UNAVAILABLE; McLean (VA); On-site; USD 180,000 - 240,000; Full time

Job summary

UNAVAILABLE is seeking an experienced Senior LLMOps Engineer to design and maintain production-grade LLM pipelines, deployment architectures, and monitoring across enterprise environments. This role spans model deployment, evaluation, optimization, and RAG pipelines.

You will lead secure GenAI infrastructure, implement CI/CD for AI workloads, integrate vector databases, and ensure governance, privacy, and cost efficiency while mentoring junior engineers.

Qualifications

• Ability to obtain and maintain a U.S. government security clearance.
• Bachelor’s degree and 10+ years of total experience.
• 5+ years in software engineering, data engineering, MLOps, or cloud engineering, with 2+ years in LLM/GenAI operations.
• Hands-on experience building RAG pipelines.
• Fluency in Python.
• Experience deploying models with HF Transformers, vLLM, TensorRT-LLM.
• Experience with FastAPI, PyTorch, LangChain, LlamaIndex, vector DBs (FAISS/Milvus/Pinecone).
• Cloud platforms (AWS/Azure/GCP) with hosting, compute, secure networking.
• CI/CD pipelines, testing for AI workloads.
• Docker, Kubernetes, Terraform/CloudFormation.
• MLSocOps, AI governance, model hardening, content safety monitoring.
• Logging/observability for high-throughput LLMs.
• Excellent written and verbal communication.
• Balance long-term platform thinking with hands-on ops.
• Experience in Agile environments.

Responsibilities

• Architect, build, and maintain LLM and RAG pipelines including hosting and retrieval layers.
• Lead secure GenAI infrastructure across cloud environments for reliability and cost efficiency.
• Build automated evaluation systems for LLM output quality, safety, latency, and governance.
• Develop CI/CD workflows for LLM/GenAI apps, including data/versioning and model lineage.
• Collaborate with AI Product Engineers and Data Scientists to productionize prototypes.
• Integrate vector databases, model gateways, content filters, and guardrails.
• Implement observability tracking performance, hallucinations, costs, and user patterns.
• Lead troubleshooting and root-cause analysis for deployment and pipeline issues.
• Develop Python-based services and integrations supporting LLM/RAG pipelines.
• Stay current with MLSecOps patterns and AI governance frameworks.

Skills

Python

Communication

Agile

Education

Bachelor's degree

Tools

Hugging Face Transformers

vLLM

TensorRT-LLM

FastAPI

PyTorch

LangChain

LlamaIndex

FAISS

Milvus

Pinecone

Docker

Kubernetes

Terraform

CloudFormation

AWS

Azure

GCP

Job description

Overview

We are looking for an experienced Senior LLMOps Engineer to design, implement, and maintain production-grade large language model (LLM) pipelines, deployment architectures, and monitoring systems across enterprise environments. The Senior LLMOps Engineer will play a critical role in operationalizing generative AI capabilities, ensuring that LLM-based applications are scalable, secure, reliable, and compliant with emerging AI risk and governance frameworks. This role spans model deployment, orchestration, evaluation, optimization, and Retrieval-Augmented Generation (RAG) pipelines.

Responsibilities

• Architect, build, and maintain scalable LLM and Retrieval-Augmented Generation (RAG) pipelines, including model hosting, inference optimization, retrieval layers, and context management frameworks.

• Lead the design and implementation of secure GenAI infrastructure across cloud environments, ensuring reliability, performance, scalability, and cost efficiency.

• Build and manage automated evaluation systems that assess LLM output quality, safety, latency, and adherence to AI governance requirements.

• Develop CI/CD workflows tailored for LLM- and GenAI-based applications, including dataset versioning, model lineage, and automated testing of prompt and model behaviors.

• Collaborate with AI Product Engineers and Data Scientists to productionize LLM-based prototypes into enterprise-grade, maintainable systems.

• Integrate vector databases, model gateways, content filters, and guardrail frameworks into end-to-end LLM and RAG solutions.

• Implement observability and monitoring solutions that track performance metrics, hallucination rates, cost profiles, and user interaction patterns.

• Lead troubleshooting and root-cause analysis for issues related to LLM deployment, inference performance, retrieval pipelines, or overall pipeline reliability.

• Develop and maintain Python-based services, integrations, and automation supporting LLM and RAG pipelines.

• Stay current with emerging LLM architectures, inference optimizations, fine-tuning techniques, and relevant MLSecOps patterns.

• Ensure compliance with data privacy, ethical AI, security, and AI governance frameworks throughout pipeline design and operations.

• Mentor junior engineers and contribute to AI engineering best practices, tooling, and reusable infrastructure patterns.

• You will contribute to the growth of our AI & Data Exploitation Practice!

Qualifications

• Ability to obtain and maintain a U.S. government security clearance.

• Bachelor’s degree and 10+ years of total experience.

• 5+ years of experience in software engineering, data engineering, MLOps, or cloud engineering, with 2+ years focused specifically on LLM or GenAI operations.

• Hands-on experience designing, building, and maintaining Retrieval-Augmented Generation (RAG) pipelines.

• Fluency in Python.

• Strong experience deploying models using frameworks such as Hugging Face Transformers, vLLM, TensorRT-LLM, or similar.

• Experience with operational tooling and frameworks such as FastAPI, PyTorch, LangChain, LlamaIndex, and vector databases such as FAISS, Milvus, Pinecone, or similar.

• Advanced knowledge of cloud platforms such as AWS, Azure, or GCP, including model hosting, distributed compute, and secure networking patterns.

• Hands-on experience building CI/CD pipelines, automated testing frameworks, and environment provisioning for AI/ML workloads.

• Experience with Docker, Kubernetes, and infrastructure-as-code tools such as Terraform or CloudFormation.

• Familiarity with MLSecOps, AI governance, model hardening, prompt injection defenses, and content safety monitoring.

• Strong understanding of logging, observability, and performance profiling for high-throughput LLM inference systems.

• Excellent written and verbal communication skills, with the ability to explain trade-offs and architectural decisions to technical and non-technical stakeholders.

• Demonstrated ability to balance long-term platform thinking with hands-on operations and rapid problem solving.

• Experience working in Agile development environments and using modern project management tools.