LLM / GenAI Engineer

Evlo AI; Minneapolis (MN); On-site; USD 140,000 - 190,000; Full time

Job summary

Evlo AI seeks an engineer to join a core AI product team building production LLM systems, including RAG pipelines, vector DBs, and agentic workflows. You will work with backend engineers, scientists, and product leaders to move prototypes to dependable production, with focus on latency, cost, and reliability.

You will design scalable retrieval-augmented systems, tune embeddings, and implement evaluation frameworks while applying LoRA/QLoRA techniques across GPUs and cloud environments.

Qualifications

• 3-6 years software eng. with 2+ years shipping LLM/GenAI features.
• Strong Python, async programming, API design, and clean architecture.
• Production experience with RAG systems and vector databases.
• Knowledge of transformer tech, embeddings, open-weight vs API-hosted models.
• Experience deploying services on cloud platforms with Docker and GPUs.
• Bachelor's degree or equivalent experience; bonus for MLOps or Kubernetes.

Responsibilities

• Design, build, and maintain production RAG pipelines and context assembly.
• Tune vector DBs (Pinecone/Weaviate/pgvector) and embedding models at scale.
• Create agentic workflows with tool calling and orchestration using LangGraph or APIs.
• Develop LLM evaluation frameworks and regression suites.
• Perform LoRA/QLoRA fine-tuning and manage GPU training runs.
• Optimize inference cost and latency with quantization, batching, and routing.
• Instrument systems with tracing and observability; support on-call quality.

Skills

Python

LLM/GenAI

APIs (FastAPI)

Distributed systems

Docker

GPU inference

Observability

Education

Bachelor's degree in CS or related

Tools

LangChain

LlamaIndex

Pinecone

Weaviate

pgvector

LangGraph

OpenTelemetry

OpenAI/Anthropic APIs

Job description

About The Role

The role sits at the core of an AI product team building production LLM systems - retrieval-augmented generation pipelines, agentic workflows, and fine-tuned models deployed to enterprise-scale workloads. This is hands-on engineering: latency budgets, cost-per-token, hallucination mitigation, and evaluation harnesses that catch regressions before clients do.

The impact is direct. GenAI features shipped by this team are customer-facing and revenue-linked, and the role works alongside backend engineers, applied scientists, and product leadership to take LLM prototypes from demo to dependable production systems.

Key Responsibilities

• Design, build, and maintain production RAG pipelines - chunking strategies, hybrid retrieval (dense + BM25), reranking, and context assembly - using LangChain, LlamaIndex, or custom architectures
• Integrate and tune vector databases (Pinecone, Weaviate, pgvector) for semantic search at scale, including embedding model selection and reindexing strategies
• Build agentic workflows with tool calling, function orchestration, and guardrails using frameworks like LangGraph or OpenAI/Anthropic tool APIs
• Develop systematic LLM evaluation frameworks: golden datasets, LLM-as-judge pipelines, regression suites, and hallucination/faithfulness metrics
• Run parameter-efficient fine-tuning (LoRA, QLoRA) and prompt optimization on domain‑specific datasets; manage training runs on GPU infrastructure
• Optimize inference cost and latency through quantization, caching layers, batching, and model routing across OpenAI, Anthropic, and open-weight models (vLLM, Llama)
• Instrument production LLM systems with tracing, observability, and alerting (LangSmith, OpenTelemetry, custom dashboards); own on‑call quality for AI services

What We Are Looking For

• 3-6 years of software engineering experience, with at least 2 years building and shipping LLM/GenAI features to production
• Strong Python engineering skills: async programming, API design (FastAPI or equivalent), testing, and clean service architecture
• Hands-on production experience with RAG systems and vector databases - not just prototypes
• Working knowledge of transformer architecture, tokenization, embeddings, and the practical tradeoffs of open-weight vs. API-hosted models
• Experience deploying and operating services in AWS, GCP, or Azure, including Docker, CI/CD, and GPU-based inference
• Bachelor's degree in Computer Science, Engineering, or related quantitative field, or equivalent practical experience
• Bonus: experience fine-tuning models (LoRA/QLoRA) with Hugging Face + PyTorch; contributions to open-source LLM tooling; MLOps experience (Kubernetes, Ray, MLflow); SQL and data pipeline experience