LLM / GenAI Engineer

Evlo AI; Miami (FL); On-site; USD 120,000 - 180,000; Full time

Job summary

Evlo AI is seeking an ML/Software Engineer to design, build, and ship production-grade LLM systems. You will work on retrieval-augmented-generation pipelines, vector databases, and agent-like workflows, collaborating with researchers, backend engineers, and product leaders.

Ideal candidates have 3–6 years in software/ML engineering, strong Python skills, and hands-on experience with RAG architectures, model fine-tuning, and cloud deployments.

Qualifications

• 3–6 years of software/ML engineering, with at least 2 years shipping LLM-powered systems to production.
• Strong Python engineering skills including async programming, API design, and testing.
• Hands-on production experience with RAG architecture, embedding models, vector DBs, and retrieval quality tuning.
• Experience fine-tuning or evaluating open-source models (Llama, Mistral, Quwen) and using provider APIs (OpenAI, Anthropic, Bedrock).

Responsibilities

• Architect and build RAG pipelines using LangChain, LlamaIndex, or custom retrieval stacks.
• Implement and scale vector database integrations for low-latency semantic search in production.
• Design agentic workflows with tool calling, function execution, and guardrails with LangGraph or custom layers.
• Run fine-tuning workloads (LoRA, QLoRA, DPO) on domain datasets using PyTorch and Hugging Face.
• Build systematic LLM evaluation frameworks with golden datasets and CI integration.
• Deploy and serve models via vLLM, Triton, or managed endpoints, optimizing cost and latency.
• Own observability for GenAI features with tracing and dashboards to surface failures.

Skills

Python

Async programming

API design

Testing

Large codebases

Education

Bachelors degree in CS/Engineering or related field

Tools

LangChain

LlamaIndex

Pinecone

Weaviate

pgvector

PyTorch

Hugging Face

OpenAI API

AWS

GCP

Azure

LoRA

QLoRA

DPO

LangGraph

Job description

About The Role

The role designs and ships production LLM and GenAI systems: retrieval-augmented generation pipelines, agentic workflows, fine-tuning jobs, and evaluation harnesses that hold up under real enterprise traffic.

This is not a prompt-tinkering position. The team builds the core AI platform — model serving infrastructure, evaluation frameworks, and RAG architecture — and this role owns meaningful pieces of it end-to-end, working alongside applied scientists, backend engineers, and product leadership.

Key Responsibilities

• Architect and build RAG pipelines using LangChain, LlamaIndex, or custom retrieval stacks, tuning chunking strategies, hybrid search, and reranking for measurable quality gains
• Implement and scale vector database integrations (Pinecone, Weaviate, pgvector) serving low-latency semantic search in production
• Design agentic workflows with tool calling, function execution, and guardrails using frameworks like LangGraph or custom orchestration layers
• Run fine-tuning workloads (LoRA, QLoRA, DPO) on domain-specific datasets using PyTorch and Hugging Face, managing GPU resources across training runs
• Build systematic LLM evaluation frameworks: golden datasets, LLM-as-judge pipelines, and regression suites wired into CI so quality regressions are caught before deploy
• Deploy and serve models via vLLM, Triton, or managed inference endpoints, optimizing for cost, latency, and throughput
• Own observability for GenAI features: tracing with LangSmith or OpenTelemetry, drift detection, and dashboards that surface failure modes quickly

What We Are Looking For

• 3–6 years of software engineering or ML engineering experience, with at least 2 years shipping LLM-powered systems to production
• Strong Python engineering skills: async programming, API design, testing, and working comfortably in large codebases
• Hands-on production experience with RAG architecture, including embedding model selection, vector databases, and retrieval quality tuning
• Experience fine-tuning or evaluating open-source models (Llama, Mistral, Qwen) and working with provider APIs (OpenAI, Anthropic, Bedrock)
• Working knowledge of cloud infrastructure (AWS, GCP, or Azure), containerized deployments, and GPU-based serving
• Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience
• Bonus: experience with RLHF or preference optimization, contributions to open-source LLM tooling, multimodal model experience, or an MS/PhD in a relevant field