Senior AI Platform Engineer: Open-Weight ML Infra & APIs

Scrums.com; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Scrums.com is seeking a hands-on senior backend engineer to build and operate the Python services and infrastructure that power open-weight language models running in private, customer-controlled environments.

You will own design, delivery, and operations across APIs, microservices, data pipelines, and GPU/CPU inference, using FastAPI, Kafka, MongoDB, and Azure, with a focus on reliability and security.

Qualifications

• At least seven years of professional software engineering experience, including strong production Python work.
• Experience building production APIs or services with FastAPI or a comparable framework.
• Hands-on experience with Kafka, event-driven architecture, and MongoDB.
• Experience with Azure and containerized deployments.
• Experience deploying open-weight or local language models and building model-serving or inference APIs.
• Familiarity with model versioning, language-model evaluation, GPU or CPU inference, observability, and performance tuning.

Responsibilities

• Develop and maintain production Python APIs and services using FastAPI or a similar framework.
• Build microservices and event-driven systems with Kafka and MongoDB.
• Deploy containerized workloads on Azure and support private AI deployments subject to security and compliance needs.
• Run open-weight or local language models and provide inference APIs that return structured outputs.
• Manage model versions and evaluation, and work with GPU and CPU inference environments.
• Improve observability, performance, monitoring, and infrastructure resource use.

Skills

Python

FastAPI

APIs

Observability

Performance tuning

GPU Inference

Model Serving

Python production

Backend development

Open-weight models

Tools

Kafka

MongoDB

Azure

Containerization

Job description

Scrums.com is seeking a hands-on senior backend engineer to build and operate the Python services and infrastructure that power open-weight language models running in private, customer-controlled environments.

You will own design, delivery, and operations across APIs, microservices, data pipelines, and GPU/CPU inference, using FastAPI, Kafka, MongoDB, and Azure, with a focus on reliability and security.