Senior ML Infra Engineer – Scalable Cloud & LLM Ops

Philo; New York (NY); On-site; USD 150,000 - 210,000; Full time

Job summary

Philo is seeking an experienced Infrastructure MLOps Software Engineer to elevate our ML infrastructure used across content recommendations, analytics, and discovery. You will help scale model serving, evaluation, and platform services on Kubernetes, GCP, and AWS, while driving reliability and security in a modern cloud-native stack.

You will partner with product and engineering teams to ship robust pipelines, improve observability, and optimize cost and performance for a growing streaming

Qualifications

• 8+ years of software development and infrastructure management experience.
• 4+ years developing and deploying ML systems at scale across training and inference.
• Experience with distributed systems and microservices architecture.
• Experience with Linux and containerized environments (Docker).
• Experience managing cloud computing environments (AWS or GCP) and configuring cloud services.

Responsibilities

• Manage the core infrastructure at Philo with a focus on supporting ML and LLM workloads.
• Design, build, and maintain cloud infrastructure components (AWS and Kubernetes).
• Build and scale model serving and inference infrastructure for real-time recommendations and LLM workloads.
• Manage GPU and accelerator infrastructure on Kubernetes — scheduling, capacity and quota management.
• Design, build, and maintain monitoring and observability systems for system health and model quality.
• Build and operate model evaluation infrastructure: offline/online eval pipelines, datasets, A/B testing.
• Design, build, and maintain CI/CD systems enabling Philo engineers to deploy code.
• Establish tools and best practices for interacting with Philo ML infra; ensure reliability and security.

Skills

Golang

Python

TypeScript

C++

Tools

Kubernetes

Docker

Terraform

AWS

GCP

Job description

Philo is seeking an experienced Infrastructure MLOps Software Engineer to elevate our ML infrastructure used across content recommendations, analytics, and discovery. You will help scale model serving, evaluation, and platform services on Kubernetes, GCP, and AWS, while driving reliability and security in a modern cloud-native stack.

You will partner with product and engineering teams to ship robust pipelines, improve observability, and optimize cost and performance for a growing streaming