LLM & GenAI Production Engineer

Evlo AI; Minneapolis (MN); On-site; USD 140,000 - 190,000; Full time

Job summary

Evlo AI seeks an engineer to join a core AI product team building production LLM systems, including RAG pipelines, vector DBs, and agentic workflows. You will work with backend engineers, scientists, and product leaders to move prototypes to dependable production, with focus on latency, cost, and reliability.

You will design scalable retrieval-augmented systems, tune embeddings, and implement evaluation frameworks while applying LoRA/QLoRA techniques across GPUs and cloud environments.

Qualifications

• 3-6 years software eng. with 2+ years shipping LLM/GenAI features.
• Strong Python, async programming, API design, and clean architecture.
• Production experience with RAG systems and vector databases.
• Knowledge of transformer tech, embeddings, open-weight vs API-hosted models.
• Experience deploying services on cloud platforms with Docker and GPUs.
• Bachelor's degree or equivalent experience; bonus for MLOps or Kubernetes.

Responsibilities

• Design, build, and maintain production RAG pipelines and context assembly.
• Tune vector DBs (Pinecone/Weaviate/pgvector) and embedding models at scale.
• Create agentic workflows with tool calling and orchestration using LangGraph or APIs.
• Develop LLM evaluation frameworks and regression suites.
• Perform LoRA/QLoRA fine-tuning and manage GPU training runs.
• Optimize inference cost and latency with quantization, batching, and routing.
• Instrument systems with tracing and observability; support on-call quality.

Skills

Python

LLM/GenAI

APIs (FastAPI)

Distributed systems

Docker

GPU inference

Observability

Education

Bachelor's degree in CS or related

Tools

LangChain

LlamaIndex

Pinecone

Weaviate

pgvector

LangGraph

OpenTelemetry

OpenAI/Anthropic APIs

Job description

Evlo AI seeks an engineer to join a core AI product team building production LLM systems, including RAG pipelines, vector DBs, and agentic workflows. You will work with backend engineers, scientists, and product leaders to move prototypes to dependable production, with focus on latency, cost, and reliability.

You will design scalable retrieval-augmented systems, tune embeddings, and implement evaluation frameworks while applying LoRA/QLoRA techniques across GPUs and cloud environments.