Senior ML Infra Engineer - Kubernetes & Cloud

Philo; Cambridge (MA); Remote; USD 185,000 - 240,000; Full time

Benefits offered by this job

Full health coverage

401(k) plan with company match

Flexible working hours

Parental leave

Generous vacation & sick leave

Job summary

Philo’s infrastructure team is expanding to accelerate ML/LLM workloads across live streaming and on‑demand services. You’ll design cloud components (AWS, Kubernetes), scale model serving, and optimize latency, throughput, and cost. Collaboration with product and engineering teams helps deploy robust ML pipelines, monitoring, and governance.

We value pragmatic, scalable solutions and long‑term reliability. We seek an Infrastructure MLOps Engineer who can drive best practices, own critical ML

Qualifications

• 8+ years of software development and infrastructure management experience.
• 4+ years of experience developing and deploying ML systems at scale across training and inference.
• Experience with distributed systems and microservices architecture principles.
• Experience with Linux and containerized environments (Docker).
• Experience managing cloud environments (AWS or GCP) and configuring services.
• Experience with Kubernetes and GPU/accelerated workloads.
• Strong understanding of networking, security, and IaC (Terraform, CloudFormation).
• Familiarity with monitoring/observability tools (Prometheus, Grafana, Datadog).
• Familiarity with CI/CD automation systems (Jenkins, GitLab).
• Experience with relational and non-relational databases and data warehousing.

Responsibilities

• Manage infrastructure for ML/LLM workloads in Philo's production environment.
• Design and maintain cloud components (AWS, Kubernetes), balance performance and cost.
• Scale model serving and inference infrastructure for real-time and ML workloads.
• Manage GPU/accelerator infrastructure on Kubernetes, including scheduling and quotas.
• Build monitoring/observability systems for system health and model quality.
• Develop model evaluation pipelines, offline/online eval, A/B testing, feedback loops.
• Develop CI/CD pipelines to automate testing and deployment of code.
• Establish IaC practices and tooling for reliability and security across Philo infra.
• Collaborate with engineers to deploy and instrument software systems.
• Evaluate and integrate third-party vendor systems for ML infrastructure.

Skills

Golang

TypeScript

Python

C++

Distributed systems

LLM/MLOps concepts

Networking basics

Tools

Kubernetes

Docker

AWS

GCP

Terraform

CloudWatch

Prometheus

Job description

Philo’s infrastructure team is expanding to accelerate ML/LLM workloads across live streaming and on‑demand services. You’ll design cloud components (AWS, Kubernetes), scale model serving, and optimize latency, throughput, and cost. Collaboration with product and engineering teams helps deploy robust ML pipelines, monitoring, and governance.

We value pragmatic, scalable solutions and long‑term reliability. We seek an Infrastructure MLOps Engineer who can drive best practices, own critical ML