Senior LLM & GenAI Platform Engineer

Evlo AI; Dallas (TX); On-site; USD 120,000 - 180,000; Full time

Job summary

Evlo AI is seeking an engineer who has shipped LLM applications to real users and knows production GenAI is 20% prompting and 80% evaluation, retrieval quality, latency, and cost control. You will own significant components of the AI platform end-to-end and collaborate with applied scientists, product engineers, and leadership to decide what to build and ship.

You will design production RAG pipelines, build LLM evaluation frameworks, fine-tune open-weight models, and deploy agentic workflows

Qualifications

• 3–6 years of software engineering experience with 2+ years shipping LLM/GenAI in production.
• Hands-on experience with RAG architectures: embedding selection, chunking, reranking, evaluation.
• Strong Python engineering: async, API design, testing; knowledge of PyTorch/Hugging Face.

Responsibilities

• Design and build production RAG pipelines with hybrid retrieval and reranking.
• Develop LLM evaluation frameworks, automated tests, and online quality monitoring.
• Fine-tune open-weight models (Llama, Mistral, Qwen) using LoRA/QLoRA; manage runs and versions.
• Develop agentic workflows with tool calling, structured outputs, and multi-step reasoning.
• Optimize inference cost and latency via routing, caching, quantization, and prompt compression.
• Ship and maintain serving infra using FastAPI, vLLM/TGI, Docker, Kubernetes on AWS or GCP.
• Lead architecture reviews and mentor engineers on LLM engineering practices.

Education

Bachelor's degree in CS/engineering or equivalent

Tools

LangChain

LlamaIndex

Pinecone

Weaviate

pgvector

Docker

Kubernetes

FastAPI

vLLM/TGI

Job description

Evlo AI is seeking an engineer who has shipped LLM applications to real users and knows production GenAI is 20% prompting and 80% evaluation, retrieval quality, latency, and cost control. You will own significant components of the AI platform end-to-end and collaborate with applied scientists, product engineers, and leadership to decide what to build and ship.

You will design production RAG pipelines, build LLM evaluation frameworks, fine-tune open-weight models, and deploy agentic workflows