AI Engineer: LLMs, RAG & MCP on Azure

Salvo Software LLC; United States; Hybrid; USD 110,000 - 180,000; Full time

Job summary

Salvo Software LLC is expanding its AI department and seeks a hands-on AI Developer to build and operate LLM-powered systems. You will serve open-source models with Ollama and llama.cpp, develop RAG pipelines, and create MCP integrations that connect LLMs to internal tools and data sources.

You will implement production-grade Python services for ML inference, work with DevOps practices, and grow toward model optimization including quantization and GPU tuning.

Qualifications

• 3–5+ years engineering experience with production-level Python for building real services, LLM inference, RAG pipelines, and automated tooling.
• Hands-on LLM & AI integration serving models via Ollama/llama.cpp, implementing vector retrieval/RAG, and developing MCP integrations (GPU/CUDA tuning & model quantization nice to have).
• Cloud & Infrastructure expertise covering Azure/AWS deployments, Docker containerization, Linux/Windows admin, and CI/CD automation (Azure DevOps, GitHub Actions, or GitLab CI), with Terraform, Kubernetes, Kafka, or systems languages (Go, Rust, C++) as a plus.
• DevOps, Security & Operations awareness, including Git workflows, scripting (Python, Bash, PowerShell), observability tools (Prometheus/Grafana), DevSecOps, and compliance frameworks (SOC 2, ISO 27001, Zero Trust).
• Strong problem-solving & communication with a root-cause troubleshooting mindset, self-directed learning, strong ownership, and seamless remote/distributed team collaboration.

Responsibilities

• AI / LLM Engineering (core focus)
• Serve and operate open-source LLMs using Ollama and llama.cpp, locally and in on-prem environments.
• Build and maintain RAG pipelines: embeddings, vector databases, chunking strategies, and retrieval quality.
• Develop MCP (Model Context Protocol) integrations connecting LLMs to internal tools and data sources.
• Build backend services in Python that power ML inference and AI-driven product features.
• Parse and process structured and semi-structured data (XML/XSD, Office document formats) as pipeline inputs.
• Grow into model optimization over time: quantization (GGUF), GPU/CUDA tuning, and offline/air-gapped deployments.
• DevOps & Infrastructure (supporting)
• Build and maintain CI/CD pipelines for AI services (Azure DevOps preferred; GitHub Actions / GitLab CI also used).
• Deploy AI workloads to Microsoft Azure,AWS and containerize services with Docker.
• Automate operational tasks with Python, Bash and/or PowerShell scripting.
• Troubleshoot across the stack - dig into root causes rather than patching symptoms.
• Support event-driven architectures using Apache Kafka (producers/consumers).
• 3–5+ years engineering experience with production-level Python for building real services, LLM inference, RAG pipelines, and automated tooling.

Skills

Python

LLM integration

RAG pipelines

Open-source LLMs

DevOps

Docker

Azure

Kubernetes

CI/CD

Quantization

Tools

Ollama

llama.cpp

Docker

Kubernetes

Terraform

Prometheus

Grafana

Azure DevOps

GitHub Actions

GitLab CI

Kafka

PowerShell

Job description

Salvo Software LLC is expanding its AI department and seeks a hands-on AI Developer to build and operate LLM-powered systems. You will serve open-source models with Ollama and llama.cpp, develop RAG pipelines, and create MCP integrations that connect LLMs to internal tools and data sources.

You will implement production-grade Python services for ML inference, work with DevOps practices, and grow toward model optimization including quantization and GPU tuning.