LLM / GenAI Engineer

Evlo AI; Chicago (IL); On-site; USD 140,000 - 210,000; Full time

Job summary

Evlo AI is seeking a hands-on engineer to build LLM-powered systems at the core of the product, including retrieval pipelines, agentic workflows, and evaluation infrastructure. You’ll own systems end‑to‑end, ship code daily, and influence architecture with production impact.

The role requires 3–7 years of software experience, strong Python, and practical GenAI deployment knowledge on cloud platforms. You will work on complex, production‑grade GenAI features with direct operational visibility.

Qualifications

• 3–7 years of software engineering experience, incl. 2+ years in production GenAI.
• Strong Python skills, experience with async programming and API design.
• Hands-on with vector DBs and embedding model stacks in real deployments.

Responsibilities

• Design and implement end‑to‑end RAG pipelines and context optimization.
• Scale vector DB integrations handling millions of documents with sub‑second latency.
• Develop agentic workflows with tool-calling and multi-step reasoning.

Skills

Python

Async programming

API design

Production‑grade software

LLM/GenAI systems

Vector database

Embedding model stack

Tokenization

Prompt engineering

Deployment on cloud

Docker

CI/CD

Education

Bachelor's degree in CS/Engineering

Tools

Pinecone

Weaviate

pgvector

Docker

FastAPI

PyTorch

Hugging Face

Job description

About The Role

The role builds the LLM-powered systems at the core of the product: retrieval pipelines, agentic workflows, and evaluation infrastructure that determine whether AI features actually work in production.

About The Role

The role builds the LLM-powered systems at the core of the product: retrieval pipelines, agentic workflows, and evaluation infrastructure that determine whether AI features actually work in production.

This is a hands‑on engineering position on a small, senior team — expect to ship code daily, own systems end‑to‑end, and make architecture decisions with real production consequences.

Key Responsibilities

• Design and implement RAG pipelines end‑to‑end: chunking strategies, hybrid retrieval, reranking, and context optimization using LangChain, LlamaIndex, or custom architectures
• Build and scale vector database integrations (Pinecone, Weaviate, pgvector) handling millions of documents with sub‑second retrieval latency
• Develop agentic workflows using tool‑calling, function execution, and multi‑step reasoning frameworks — deployed and monitored in production
• Own LLM evaluation infrastructure: golden datasets, LLM‑as‑judge pipelines, regression test suites, and automated quality gates in CI/CD
• Run parameter‑efficient fine‑tuning (LoRA, QLoRA) and structured output optimization on domain‑specific datasets using PyTorch and Hugging Face
• Integrate LLM applications with backend services via FastAPI, streaming responses, and robust async patterns
• Instrument observability for GenAI systems: tracing (LangSmith/Langfuse), token cost tracking, latency dashboards, and hallucination detection alerts

What We Are Looking For

• 3–7 years of software engineering experience, including at least 2 years building LLM/GenAI systems in production
• Deep proficiency in Python with experience in async programming, API design, and production‑grade software practices
• Hands‑on experience with at least one vector database and embedding model stack in a real deployment
• Practical understanding of LLM internals: tokenization, context windows, prompt engineering tradeoffs, and inference optimization (vLLM, quantization)
• Experience deploying GenAI features on AWS, GCP, or Azure, including Docker‑based containerization and CI/CD pipelines
• Bachelor's degree in Computer Science, Engineering, or equivalent practical experience
• Bonus: experience fine‑tuning open‑source models (Llama, Mistral, Qwen), prior work on agentic systems, contributions to open‑source LLM tooling, or a relevant ML publication