LLM / GenAI Engineer

Evlo AI; Dallas (TX); On-site; USD 120,000 - 180,000; Full time

Job summary

Evlo AI is seeking an engineer who has shipped LLM applications to real users and knows production GenAI is 20% prompting and 80% evaluation, retrieval quality, latency, and cost control. You will own significant components of the AI platform end-to-end and collaborate with applied scientists, product engineers, and leadership to decide what to build and ship.

You will design production RAG pipelines, build LLM evaluation frameworks, fine-tune open-weight models, and deploy agentic workflows

Qualifications

• 3–6 years of software engineering experience with 2+ years shipping LLM/GenAI in production.
• Hands-on experience with RAG architectures: embedding selection, chunking, reranking, evaluation.
• Strong Python engineering: async, API design, testing; knowledge of PyTorch/Hugging Face.

Responsibilities

• Design and build production RAG pipelines with hybrid retrieval and reranking.
• Develop LLM evaluation frameworks, automated tests, and online quality monitoring.
• Fine-tune open-weight models (Llama, Mistral, Qwen) using LoRA/QLoRA; manage runs and versions.
• Develop agentic workflows with tool calling, structured outputs, and multi-step reasoning.
• Optimize inference cost and latency via routing, caching, quantization, and prompt compression.
• Ship and maintain serving infra using FastAPI, vLLM/TGI, Docker, Kubernetes on AWS or GCP.
• Lead architecture reviews and mentor engineers on LLM engineering practices.

Education

Bachelor's degree in CS/engineering or equivalent

Tools

LangChain

LlamaIndex

Pinecone

Weaviate

pgvector

Docker

Kubernetes

FastAPI

vLLM/TGI

Job description

About The Role

The role is for an engineer who has shipped LLM applications to real users and knows that production GenAI is 20% prompting and 80% evaluation, retrieval quality, latency, and cost control. The team builds LLM-powered products serving thousands of concurrent users — RAG systems, agentic workflows, and fine-tuned domain models — where hallucination rates and p95 latency are tracked metrics, not talking points.

About The Role

The role is for an engineer who has shipped LLM applications to real users and knows that production GenAI is 20% prompting and 80% evaluation, retrieval quality, latency, and cost control. The team builds LLM-powered products serving thousands of concurrent users — RAG systems, agentic workflows, and fine-tuned domain models — where hallucination rates and p95 latency are tracked metrics, not talking points.

You will own significant components of the AI platform end-to-end and work directly with applied scientists, product engineers, and leadership to decide what gets built, how it gets evaluated, and when it ships.

Key Responsibilities

• Design and build production RAG pipelines — chunking strategies, hybrid retrieval, reranking — using LangChain, LlamaIndex, or custom architectures backed by vector databases (Pinecone, Weaviate, pgvector)
• Build systematic LLM evaluation frameworks: golden datasets, LLM-as-judge pipelines, automated regression testing, and online quality monitoring
• Fine-tune open-weight models (Llama, Mistral, Qwen) using LoRA/QLoRA on domain-specific datasets; manage training runs, evals, and model versioning
• Develop and deploy agentic workflows with tool calling, structured outputs, and multi-step reasoning — including guardrails, retries, and fallback strategies
• Optimize inference cost and latency through model routing, caching, quantization, and prompt compression
• Ship and maintain serving infrastructure using FastAPI, vLLM or TGI, Docker, and Kubernetes on AWS or GCP
• Write observable, tested, well-documented code; lead architecture reviews and mentor engineers on LLM engineering practices

What We Are Looking For

• 3–6 years of software engineering experience, including 2+ years building LLM/GenAI systems in production
• Deep hands‑on experience with RAG architectures: embedding selection, chunking, reranking, retrieval evaluation, and debugging retrieval failures
• Strong Python engineering skills — async programming, API design, testing — plus working knowledge of PyTorch and the Hugging Face ecosystem
• Production experience with at least one vector database and one major cloud provider (AWS, GCP, or Azure)
• Practical understanding of LLM evaluation: building benchmark sets, detecting regressions, and reasoning about hallucination and retrieval quality
• Bachelor's degree in CS, engineering, or equivalent practical experience shipping ML‑backed systems
• Bonus: Experience with vLLM/TGI inference optimization, RLHF or DPO pipelines, quantization (AWQ, GPTQ), or contributions to open‑source LLM tooling; MS/PhD in a relevant field