AI Engineer — LLM, RAG & MCP Systems

Salvo Software LLC; Washington (District of Columbia); Remote; USD 120,000 - 180,000; Full time

Job summary

Salvo Software LLC is building a new AI department in the United States and seeks an hands‑on AI Developer to design and operate LLM‑powered systems. You will serve open‑source models with Ollama/llama.cpp, craft RAG pipelines, and implement MCP integrations while refining ML inference services in Python.

You will collaborate with distributed teams across the US and beyond, applying DevOps practices, Docker and cloud deployments (Azure/AWS) to deploy reliable AI features and maintain system

Qualifications

• 3–5+ years engineering experience with production‑level Python for building real services, LLM inference, RAG pipelines, and automated tooling.
• Hands‑on LLM & AI integration serving models via Ollama/llama.cpp, implementing vector retrieval/RAG, and developing MCP integrations (GPU/CUDA tuning & model quantization nice to have).
• Cloud & Infrastructure expertise covering Azure/AWS deployments, Docker containerization, Linux/Windows admin, and CI/CD automation (Azure DevOps, GitHub Actions, or GitLab CI), with Terraform, Kubernetes, Kafka, or systems languages (Go, Rust, C++) as a plus.
• DevOps, Security & Operations awareness, including Git workflows, scripting (Python, Bash, PowerShell), observability tools (Prometheus/Grafana), DevSecOps, and compliance frameworks (SOC 2, ISO 27001, Zero Trust).
• Strong problem‑solving & communication with a root‑cause troubleshooting mindset, self‑directed learning, strong ownership, and seamless remote/distributed team collaboration.

Responsibilities

• Serve and operate open‑source LLMs using Ollama and llama.cpp, locally and in on‑prem environments.
• Build and maintain RAG pipelines: embeddings, vector databases, chunking strategies, and retrieval quality.
• Develop MCP (Model Context Protocol) integrations connecting LLMs to internal tools and data sources.
• Build backend services in Python that power ML inference and AI‑driven product features.
• Parse and process structured and semi‑structured data (XML/XSD, Office document formats) as pipeline inputs.
• Grow into model optimization over time: quantization (GGUF), GPU/CUDA tuning, and offline/air‑gapped deployments.

Skills

Python

LLM integration

RAG pipelines

DevOps

Docker

Azure

AWS

CI/CD

Kubernetes

Kafka

Ollama/llama.cpp

GPU/CUDA tuning

Model quantization

Terraform

Go/Rust/C++

Tools

Docker

Azure DevOps

GitHub Actions

GitLab CI

Kubernetes

Terraform

Ollama

llama.cpp

Kafka

Prometheus/Grafana

Job description

Salvo Software LLC is building a new AI department in the United States and seeks an hands‑on AI Developer to design and operate LLM‑powered systems. You will serve open‑source models with Ollama/llama.cpp, craft RAG pipelines, and implement MCP integrations while refining ML inference services in Python.

You will collaborate with distributed teams across the US and beyond, applying DevOps practices, Docker and cloud deployments (Azure/AWS) to deploy reliable AI features and maintain system