LLM / GenAI Engineer

Evlo AI; Miami (FL); On-site; USD 140,000 - 180,000; Full time

Job summary

Evlo AI is hiring an ML/Software Engineer to design and ship production LLM and GenAI systems, including RAG pipelines, agentic workflows, and fine-tuning jobs for enterprise traffic.

You will own core platform pieces—model serving, evaluation frameworks, and RAG architecture—working with applied scientists, backend engineers, and product leadership.

Qualifications

• 3–6 years of software or ML engineering experience.
• 2+ years shipping LLM-powered systems to production.
• Strong Python, async programming, API design, and testing.
• Experience with RAG architecture, embeddings, and vector databases.
• Experience fine-tuning or evaluating open-source models and provider APIs.
• Bachelor's degree in CS/Engineering or equivalent practical experience.

Responsibilities

• Design and ship production LLM/GenAI systems and evaluation harnesses.
• Build RAG pipelines and tuning for quality gains.
• Scale vector DB integrations for low-latency search.
• Develop agentic workflows with tool calling and guardrails.
• Run fine-tuning workloads on domain data using PyTorch/Hugging Face.
• Build LLM evaluation frameworks and CI pipelines.
• Deploy models via inference endpoints and optimize cost/latency.

Skills

Python

Async programming

LLM engineering

RAG pipelines

Education

Bachelor's degree in CS/Engineering

Tools

LangChain

LlamaIndex

Pinecone

Weaviate

pgvector

PyTorch

Hugging Face

OpenAI API

Job description

About The Role

The role designs and ships production LLM and GenAI systems: retrieval-augmented generation pipelines, agentic workflows, fine-tuning jobs, and evaluation harnesses that hold up under real enterprise traffic.

About The Role

The role designs and ships production LLM and GenAI systems: retrieval-augmented generation pipelines, agentic workflows, fine-tuning jobs, and evaluation harnesses that hold up under real enterprise traffic. This is not a prompt-tinkering position. The team builds the core AI platform — model serving infrastructure, evaluation frameworks, and RAG architecture — and this role owns meaningful pieces of it end-to-end, working alongside applied scientists, backend engineers, and product leadership.

Key Responsibilities

• Architect and build RAG pipelines using LangChain, LlamaIndex, or custom retrieval stacks, tuning chunking strategies, hybrid search, and reranking for measurable quality gains
• Implement and scale vector database integrations (Pinecone, Weaviate, pgvector) serving low-latency semantic search in production
• Design agentic workflows with tool calling, function execution, and guardrails using frameworks like LangGraph or custom orchestration layers
• Run fine-tuning workloads (LoRA, QLoRA, DPO) on domain-specific datasets using PyTorch and Hugging Face, managing GPU resources across training runs
• Build systematic LLM evaluation frameworks: golden datasets, LLM-as-judge pipelines, and regression suites wired into CI so quality regressions are caught before deploy
• Deploy and serve models via vLLM, Triton, or managed inference endpoints, optimizing for cost, latency, and throughput
• Own observability for GenAI features: tracing with LangSmith or OpenTelemetry, drift detection, and dashboards that surface failure modes quickly

What We Are Looking For

• 3–6 years of software engineering or ML engineering experience, with at least 2 years shipping LLM-powered systems to production
• Strong Python engineering skills: async programming, API design, testing, and working comfortably in large codebases
• Hands-on production experience with RAG architecture, including embedding model selection, vector databases, and retrieval quality tuning
• Experience fine-tuning or evaluating open-source models (Llama, Mistral, Qwen) and working with provider APIs (OpenAI, Anthropic, Bedrock)
• Working knowledge of cloud infrastructure (AWS, GCP, or Azure), containerized deployments, and GPU-based serving
• Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience
• Bonus: experience with RLHF or preference optimization, contributions to open-source LLM tooling, multimodal model experience, or an MS/PhD in a relevant field