AI Platform Engineer: Scalable, Secure, Self-Service

Machine Learning Technologies, LLC; Boston (MA); On-site; USD 150,000 - 190,000; Full time

Job summary

Machine Learning Technologies, LLC is seeking a Platform Engineer to build and maintain the compute environment for AI workloads, including GPU-backed resources and containerized infrastructure. You will design automated rollout pipelines with rollbacks, connect models to real data, and manage vector stores and guarded access across the organization.

You will ensure data privacy, security, and cost discipline while enabling self-service across teams.

Qualifications

• Platform engineering with 4+ years in cloud infrastructure or DevOps.
• AI/LLM platforms in production with data-grounded retrieval systems.
• Ability to test, evaluate, and monitor AI systems for issues.
• Strong Python skills and API integration experience.
• Cloud experience (AWS/Azure) with containerized workloads.
• Understanding cost vs performance of AI models.
• Experience exposing metrics via APIs or dashboards.
• Security-first mindset; least-privilege access and data handling.
• Ability to build self-service capabilities with guardrails.
• Comfort across infrastructure, automation, integration and code.

Responsibilities

• Infrastructure and Scaling: stand up and maintain the compute environment AI workloads run on, including containerized and GPU-backed resources, so workloads run efficiently and cost-effectively
• Automate safe rollouts: build automated pipelines for rolling out AI platform updates, with rollback built in
• Build retrieval systems: connect AI models to the organization's own data so responses are grounded in real, current information
• Maintain vector stores: connect and maintain the vector stores and data sources retrieval depends on
• Route and gate model access: give people one governed way to reach approved AI models and providers, routing each request to the model that actually fits the job
• Redact sensitive data: strip sensitive or private data out of prompts before they reach a model
• Build guardrails: put guardrails in place to catch harmful, biased, or simply wrong AI output before a user sees it
• Enable self-service without losing control: set up access and permissions so people across the organization can use approved AI capabilities on their own, while the platform still enforces where the line is
• Monitor system health: watch system health, response times, and error rates, and catch problems before people have to report them
• Track usage and spend: track usage and spend at the model and feature level so cost surprises do not happen
• Apply cost discipline: apply the same cost-management discipline to AI spend that the platform team applies everywhere else
• Build self-service tools: build self-service tools and APIs so other teams can ship AI features without needing to understand the plumbing underneath
• Report on AI performance: expose usage, spend, and performance metrics through APIs or dashboards so AI costs and performance are visible across the organization

Skills

Platform engineering

AI in production

AI systems judgment

Python

Cloud (AWS/Azure)

Cost/Performance tradeoffs

Reporting dashboards

Security mindset

Self-service tooling

Full-stack familiarity

Tools

Terraform

Kubernetes

Job description

Machine Learning Technologies, LLC is seeking a Platform Engineer to build and maintain the compute environment for AI workloads, including GPU-backed resources and containerized infrastructure. You will design automated rollout pipelines with rollbacks, connect models to real data, and manage vector stores and guarded access across the organization.

You will ensure data privacy, security, and cost discipline while enabling self-service across teams.