LLM / GenAI Engineer

Evlo AI

United States

Remote

USD 140,000 - 200,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Evlo AI is building core LLM-powered systems that drive retrieval pipelines and agentic workflows in production. You will own end-to-end architecture for multi-model pipelines, evaluation infrastructure, and scalable deployments.

You will design RAG pipelines, integrate vector databases, enable tool-calling and multi-step reasoning, and ensure observability with tracing and latency dashboards. This role seeks 3–7 years of software experience, strong Python skills, and practical

Qualifications

  • 3–7 years of software engineering experience, including at least 2 years building LLM/GenAI systems in production.

Responsibilities

  • Design and implement end-to-end RAG pipelines with chunking, hybrid retrieval, reranking, and context optimization.
  • Build and scale vector database integrations (Pinecone, Weaviate, pgvector) with sub-second latency.
  • Develop agentic workflows using tool-calling, function execution, and multi-step reasoning frameworks.
  • Own LLM evaluation infrastructure: datasets, LLM-as-judge pipelines, regression tests, and CI/CD quality gates.
  • Run parameter-efficient fine-tuning (LoRA, QLoRA) and structured output optimization on domain datasets using PyTorch and Hugging Face.
  • Integrate LLM applications with backend services via FastAPI, streaming responses, and robust async patterns.
  • Instrument observability for GenAI systems: tracing (LangSmith/Langfuse), token cost tracking, latency dashboards, and hallucination alerts.

Skills

Python
Async programming
API design
Production-grade software
LLM/GenAI systems in production

Education

Bachelor's degree in Computer Science or related field

Tools

Pinecone
Weaviate
pgvector
LangChain
LlamaIndex
PyTorch
Hugging Face
FastAPI
LangSmith
Langfuse
Docker

Job description

About The Role The role builds the LLM‑powered systems at the core of the product: retrieval pipelines, agentic workflows, and evaluation infrastructure that determine whether AI features actually work in production. This is a hands‑on engineering position on a small, senior team — expect to ship code daily, own systems end‑to‑end, and make architecture decisions with real production consequences.

Key Responsibilities
  • Design and implement RAG pipelines end‑to‑end: chunking strategies, hybrid retrieval, reranking, and context optimization using LangChain, LlamaIndex, or custom architectures
  • Build and scale vector database integrations (Pinecone, Weaviate, pgvector) handling millions of documents with sub‑second retrieval latency
  • Develop agentic workflows using tool‑calling, function execution, and multi‑step reasoning frameworks — deployed and monitored in production
  • Own LLM evaluation infrastructure: golden datasets, LLM-as-judge pipelines, regression test suites, and automated quality gates in CI/CD
  • Run parameter‑efficient fine‑tuning (LoRA, QLoRA) and structured output optimization on domain‑specific datasets using PyTorch and Hugging Face
  • Integrate LLM applications with backend services via FastAPI, streaming responses, and robust async patterns
  • Instrument observability for GenAI systems: tracing (LangSmith/Langfuse), token cost tracking, latency dashboards, and hallucination detection alerts
What We Are Looking For
  • 3–7 years of software engineering experience, including at least 2 years building LLM/GenAI systems in production
  • Deep proficiency in Python with experience in async programming, API design, and production‑grade software practices
  • Hands‑on experience with at least one vector database and embedding model stack in a real deployment
  • Practical understanding of LLM internals: tokenization, context windows, prompt engineering tradeoffs, and inference optimization (vLLM, quantization)
  • Experience deploying GenAI features on AWS, GCP, or Azure, including Docker‑based containerization and CI/CD pipelines
  • Bachelor's degree in Computer Science, Engineering, or equivalent practical experience
  • Bonus: experience fine‑tuning open‑source models (Llama, Mistral, Qwen), prior work on agentic systems, contributions to open‑source LLM tooling, or a relevant ML publication
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

LLM / GenAI Engineer
LLM / GenAI Engineer

Evlo AI • Minneapolis (MN)

On-site
USD 140,000 - 190,000
LLM / GenAI Engineer
LLM / GenAI Engineer

Latitude • Seattle (WA)

On-site
USD 150,000 - 230,000
AI/ML Engineer
AI/ML Engineer

RiskForce LLC • United States

On-site
USD 120,000 - 180,000
Junior Full-Stack Developer
Junior Full-Stack Developer

ELYTS LLC • United States

Remote
USD 90,000 - 120,000
Senior AI Engineer - GenAI + Data Platform - AWS
Senior AI Engineer - GenAI + Data Platform - AWS

Compunnel Inc. • Los Angeles (CA)

On-site
USD 120,000 - 160,000
Senior Associate - AI ML Engineer
Senior Associate - AI ML Engineer

Riveron • United States

On-site
USD 120,000 - 180,000
AI Engineer (Mid - Senior Level)
AI Engineer (Mid - Senior Level)

TIFIN • Colorado

Hybrid
USD 140,000 - 190,000
Senior LLM Engineer
Senior LLM Engineer

Turing • United States

Remote
USD 26,000 - 44,000
LLM Training & Model Development Engineer
LLM Training & Model Development Engineer

InOpTra Digital Inc • United States

Remote
USD 90,000 - 120,000
Competitive salary
Opportunity for remote work
Health benefits
Senior LLM Engineer – GenAI / ML (Python, Langchain)
Senior LLM Engineer – GenAI / ML (Python, Langchain)

Executive Staff Recruiters / ESR Healthcare • United States

Remote
USD 41,000 - 77,000
Referral Bonus $150