MLOps Engineer

Evlo AI; Boston (MA); On-site; USD 130,000 - 180,000; Full time

Job summary

Evlo AI in Boston is looking for an experienced MLOps/DevOps engineer to own the infrastructure, deployment pipelines, and scalable ML systems in production. You will work with ML engineers and data scientists to ensure models scale with low latency and strong observability across cloud platforms and model serving frameworks.

The role involves implementing CI/CD, IaC tooling, monitoring for drift, and working with security teams to enforce governance and robust access controls.

Qualifications

• 3–7 years of experience in MLOps, DevOps, or ML infrastructure engineering.
• Strong Python, Docker, and Kubernetes skills.
• Hands-on with AWS, GCP, or Azure.
• Familiar with model serving frameworks and vector stores.
• Knowledge of CI/CD and IaC tools like Terraform or Ansible.
• Nice to have LLM inference optimization experience.

Responsibilities

• Design and implement scalable MLOps infrastructure using Kubernetes, Docker, Terraform, and modern CI/CD pipelines.
• Build automated model training, validation, and deployment pipelines to streamline the path from research to production.
• Deploy and manage model serving endpoints using Triton, TorchServe, vLLM, or AWS SageMaker for low-latency inference.
• Configure automated monitoring systems for data drift, concept drift, system latency, and infrastructure resource utilization.
• Implement comprehensive logging, tracing, and observability frameworks for complex LLM and traditional ML workflows.
• Collaborate with security and engineering teams to ensure compliance, model governance, and robust access controls.

Skills

Python

Docker

Kubernetes

CI/CD

Infrastructure as Code

Cloud (AWS/GCP/Azure)

Education

Bachelor's degree in CS or related field

Tools

Feast

Hopsworks

vLLM

TorchServe

SageMaker

Terraform

Ansible

Job description

About The Role The role owns the infrastructure, orchestration, and deployment pipelines that power large-scale machine learning and generative AI systems in production.

The team works closely with machine learning engineers and data scientists to ensure models scale reliably, maintain low latency, and remain observable under heavy enterprise workloads.

Key Responsibilities

• Design and implement scalable MLOps infrastructure using Kubernetes, Docker, Terraform, and modern CI/CD pipelines
• Build automated model training, validation, and deployment pipelines to streamline the path from research to production
• Deploy and manage model serving endpoints using Triton, TorchServe, vLLM, or AWS SageMaker for low-latency inference
• Configure automated monitoring systems for data drift, concept drift, system latency, and infrastructure resource utilization
• Implement comprehensive logging, tracing, and observability frameworks for complex LLM and traditional ML workflows
• Collaborate with security and engineering teams to ensure compliance, model governance, and robust access controls

What We Are Looking For

• 3-7 years of experience in MLOps, DevOps, or machine learning infrastructure engineering
• Strong proficiency in Python, containerization technologies (Docker), and orchestration platforms (Kubernetes)
• Hands-on experience with cloud infrastructure providers such as AWS, GCP, or Azure
• Familiarity with model serving frameworks, feature stores (Feast, Hopsworks), and vector databases
• Solid understanding of CI/CD principles and infrastructure-as-code tools like Terraform or Ansible
• Bonus: Experience managing LLM inference optimization, vLLM, Triton, or large-scale distributed training clusters