LLM / GenAI Engineer

Evlo AI; Los Angeles (CA); On-site; USD 130,000 - 230,000; Full time

Job summary

Evlo AI is building production-grade LLM and GenAI systems in Los Angeles, focusing on retrieval-augmented generation, agentic workflows, fine-tuning, and scalable evaluation infrastructure. This hands-on role owns platform components end-to-end—from model selection and prompt architecture to latency, cost, and reliability in production.

You will design RAG pipelines, implement agentic features behind robust APIs, tune models with PEFT techniques, and build evaluation frameworks to ensure

Qualifications

• 3–6 years of software engineering experience.
• 1–2 years shipping LLM/GenAI systems to production.
• Strong Python engineering skills with API design experience.
• Hands-on with at least one major cloud (AWS, GCP, or Azure) and containerized deployment (Docker, Kubernetes).
• Practical knowledge of transformers, embeddings, tokenization, and tradeoffs between hosted APIs and self-hosted models.

Responsibilities

• Design and build RAG pipelines using LangChain, LlamaIndex, or custom architectures; manage chunking, hybrid retrieval, and reranking.
• Develop agentic workflows with tool calling, planning, and multi-step reasoning behind robust APIs with latency and cost budgets.
• Run parameter-efficient fine-tuning (LoRA, QLoRA, DPO) on domain datasets; manage experiment tracking and model registries.
• Build systematic LLM evaluation frameworks: golden datasets, LLM-as-judge pipelines, offline benchmarking, and pre-deployment testing.
• Own vector database infrastructure (Pinecone, Weaviate, pgvector, Qdrant) including embedding refresh and index tuning.
• Optimize inference performance: quantization, batching, caching, and serving via vLLM, TGI, or managed APIs.
• Collaborate with product and applied science teams to translate ambiguous business problems into scoped LLM features.

Skills

Python

Async programming

API design

FastAPI

AWS

GCP

Azure

Docker

Kubernetes

Transformers

Embeddings

Tokenization

Vector databases

Semantic search

Education

BS in Computer Science

MS in Computer Science

Tools

LangChain

LlamaIndex

Pinecone

Weaviate

Qdrant

vLLM

Job description

About The Role

The role focuses on building and shipping production LLM and GenAI systems: retrieval-augmented generation pipelines, agentic workflows, fine-tuning, and the evaluation infrastructure required to prove those systems actually work at scale.

About The Role

The role focuses on building and shipping production LLM and GenAI systems: retrieval-augmented generation pipelines, agentic workflows, fine-tuning, and the evaluation infrastructure required to prove those systems actually work at scale.

This is a hands‑on engineering position on a well‑funded AI team — expect to own significant components of the platform end‑to‑end, from model selection and prompt architecture to latency, cost, and reliability in production.

Key Responsibilities

• Design and build RAG pipelines using LangChain, LlamaIndex, or custom architectures, including chunking strategies, hybrid retrieval, and reranking
• Develop agentic workflows with tool calling, planning, and multi-step reasoning — deployed behind robust APIs with strict latency and cost budgets
• Run parameter-efficient fine-tuning (LoRA, QLoRA, DPO) on domain datasets; manage experiment tracking, dataset versioning, and model registries
• Build systematic LLM evaluation frameworks: golden datasets, LLM-as-judge pipelines, offline benchmarking, and pre-deployment regression testing
• Own vector database infrastructure (Pinecone, Weaviate, pgvector, or Qdrant) including embedding refresh strategies and index performance tuning
• Optimize inference performance: quantization, batching, caching, and serving via vLLM, TGI, or managed APIs
• Collaborate with product and applied science teams to translate ambiguous business problems into scoped, shippable LLM features

What We Are Looking For

• 3–6 years of software engineering experience, including at least 1–2 years shipping LLM/GenAI systems to production
• Strong Python engineering skills: async programming, API design, testing, and working with frameworks like FastAPI
• Hands‑on experience with at least one major cloud (AWS, GCP, or Azure) and containerized deployment (Docker, Kubernetes)
• Practical knowledge of transformers, embeddings, tokenization, and the tradeoffs between hosted APIs and self‑hosted open models (Llama, Mistral, Qwen)
• Experience operating vector databases and semantic search systems under real traffic
• BS/MS in Computer Science, related field, or equivalent practical experience
• Bonus: Experience with RLHF pipelines, multimodal models, structured output decoding (JSON mode, function calling at scale), or contributions to open-source LLM tooling