On-Prem LLM Engineer & RAG Specialist

Salvo Software; United States; On-site; USD 180,000 - 240,000; Full time

Job summary

Salvo Software is seeking an AI Developer with a strong backend and ML engineering background to design, train, optimize, and deploy LLM models in on-prem and offline environments. You will build end-to-end LLM pipelines, including data preprocessing, supervised fine-tuning, model quantization, evaluation, and RAG design.

You will collaborate with engineering and product teams to deploy context-aware AI systems, implement MCP server integrations, and ensure robust, air-gapped operation.

Qualifications

• Strong experience building end-to-end LLM pipelines in back-end environments.
• Hands-on experience with supervised fine-tuning and model optimization.
• Proficiency in deploying offline/on-prem LLM solutions and RAG systems.
• Familiarity with open-source models (LLaMA, Mistral, Qwen) and vector stores.

Responsibilities

• Design, train, and optimize LLMs for on-prem and air-gapped setups.
• Build and maintain RAG pipelines and document-grounded retrieval.
• Develop MCP server integrations for tool access and data sources.
• Implement and optimize data preprocessing, tokenization, and context handling.
• Collaborate with engineering and product teams on scalable backend pipelines.

Skills

Python

ML engineering

LLM development

PyTorch

TensorFlow

Backend development

MLOps

Data preprocessing

NLP/LLM fine-tuning

Tools

Docker

Git

Azure DevOps

PostgreSQL

MySQL

Job description

Salvo Software is seeking an AI Developer with a strong backend and ML engineering background to design, train, optimize, and deploy LLM models in on-prem and offline environments. You will build end-to-end LLM pipelines, including data preprocessing, supervised fine-tuning, model quantization, evaluation, and RAG design.

You will collaborate with engineering and product teams to deploy context-aware AI systems, implement MCP server integrations, and ensure robust, air-gapped operation.