MLOps Engineer

Evlo AI; New York (NY); On-site; USD 140,000 - 200,000; Full time

Job summary

Evlo AI in New York is seeking an ML infrastructure engineer to own training pipelines, model registries, CI/CD for ML, and the serving stack that keeps models fast and observable in production.

You will work at the intersection of platform engineering and machine learning, collaborating with data scientists and backend engineers to turn experimental models into dependable, high-throughput systems.

Qualifications

• 3–6 years of experience in MLOps, ML infrastructure, or backend/platform engineering supporting ML systems in production.
• Strong Python and/or Go skills; solid software engineering fundamentals including testing and code review discipline.
• Hands-on production experience with Kubernetes, Docker, and at least one major cloud (AWS, GCP, or Azure).
• Experience with ML workflow tooling: MLflow, Kubeflow, Airflow, or Vertex AI Pipelines.
• Track record of operating ML systems at scale – monitoring, incident response, and performance tuning under real traffic.
• BS in Computer Science, Engineering, or equivalent practical experience.
• Bonus: GPU orchestration experience (NVIDIA Triton, Kubernetes device plugins), feature store experience (Feast, Tecton), or contributions to open-source MLOps tooling

Responsibilities

• Build and maintain end-to-end ML pipelines (training, validation, deployment) using Kubeflow, Airflow, or Dagster
• Own the model serving infrastructure – containerized inference services on Kubernetes with GPU scheduling, autoscaling, and sub-100ms latency targets
• Design CI/CD workflows for ML: automated testing, model versioning with MLflow, canary deployments, and one-click rollback
• Implement monitoring and observability for production models: data drift detection, latency/throughput dashboards, and alerting with Prometheus and Grafana
• Manage feature stores and data pipelines to guarantee training/serving consistency across environments
• Optimize inference cost and performance – quantization, batching strategies, GPU utilization, and model compilation (TensorRT, ONNX)
• Establish MLOps standards and tooling; run postmortems and drive reliability improvements across the ML platform

Skills

Python

Go

Testing

Code review

Education

BS in Computer Science, Engineering, or equivalent

Tools

Kubernetes

Docker

AWS

GCP

Azure

MLflow

Kubeflow

Airflow

Vertex AI Pipelines

Prometheus

Grafana

Job description

About The Role

The role owns the infrastructure that makes ML possible at scale: training pipelines, model registries, CI/CD for ML, and the serving stack that keeps models fast, reliable, and observable in production.

The team sits at the intersection of platform engineering and machine learning – working directly with data scientists and backend engineers to turn experimental models into dependable, high-throughput systems.

Key Responsibilities

• Build and maintain end-to-end ML pipelines (training, validation, deployment) using Kubeflow, Airflow, or Dagster
• Own the model serving infrastructure – containerized inference services on Kubernetes with GPU scheduling, autoscaling, and sub-100ms latency targets
• Design CI/CD workflows for ML: automated testing, model versioning with MLflow, canary deployments, and one-click rollback
• Implement monitoring and observability for production models: data drift detection, latency/throughput dashboards, and alerting with Prometheus and Grafana
• Manage feature stores and data pipelines to guarantee training/serving consistency across environments
• Optimize inference cost and performance – quantization, batching strategies, GPU utilization, and model compilation (TensorRT, ONNX)
• Establish MLOps standards and tooling; run postmortems and drive reliability improvements across the ML platform

What We Are Looking For

• 3–6 years of experience in MLOps, ML infrastructure, or backend/platform engineering supporting ML systems in production
• Strong Python and/or Go skills; solid software engineering fundamentals including testing and code review discipline
• Hands-on production experience with Kubernetes, Docker, and at least one major cloud (AWS, GCP, or Azure)
• Experience with ML workflow tooling: MLflow, Kubeflow, Airflow, or Vertex AI Pipelines
• Track record of operating ML systems at scale – monitoring, incident response, and performance tuning under real traffic
• BS in Computer Science, Engineering, or equivalent practical experience
• Bonus: GPU orchestration experience (NVIDIA Triton, Kubernetes device plugins), feature store experience (Feast, Tecton), or contributions to open-source MLOps tooling