ML Infrastructure Engineer

DocVerify; United States; Remote; USD 130,000 - 190,000; Full time

Job summary

DocVerify is seeking an experienced ML infrastructure engineer to build and operate the training, versioning, deployment, and serving infrastructure for multiple computer vision and fraud-detection models. The role targets sub-200 ms API responses with auditable changes and zero-downtime releases.

You will manage model-serving on Google Cloud (GKE/Vertex AI) and implement automated pipelines, versioning, canaries, and A/B testing while optimizing latency and monitoring throughput and quality.

Qualifications

• A minimum of 3 years in ML infrastructure, MLOps, or backend systems engineering.
• Strong Kubernetes and Docker experience, plus cloud deployment work on GCP or AWS.
• Experience with a model-serving framework such as Triton, TorchServe, or TensorFlow Serving.
• Proficiency in Python and working knowledge of Go, Rust, or C++.
• Experience building ML delivery pipelines with tools such as MLflow or Kubeflow, or custom equivalents.
• Understanding of GPU inference optimization, including TensorRT or ONNX Runtime.

Responsibilities

• Develop and maintain model-serving infrastructure on Google Cloud, using services such as Cloud Run, GKE, or Vertex AI.
• Create automated pipelines for model training, evaluation, approval, and production rollout.
• Introduce model versioning, canary releases, and A/B testing.
• Reduce inference latency through batching, quantization, and hardware-aware compilation.
• Monitor model quality, drift, and request throughput, and configure alerts.
• Allocate GPU and TPU capacity across training and serving, balancing performance with cost.

Skills

Kubernetes

Docker

Python

Go

Rust

C++

Triton

TensorFlow Serving

MLflow

Tools

Kubeflow

MLflow

TorchServe

TensorFlow Serving

Vertex AI

SageMaker

Job description

Role overview

Build and operate the infrastructure that trains, versions, deploys, and serves multiple computer vision and fraud detection models. The work spans cloud systems, model delivery workflows, inference performance, and operational monitoring, with a stated target of keeping API responses under 200 ms while supporting auditable changes and zero-downtime releases.

Responsibilities

• Develop and maintain model-serving infrastructure on Google Cloud, using services such as Cloud Run, GKE, or Vertex AI.
• Create automated pipelines for model training, evaluation, approval, and production rollout.
• Introduce model versioning, canary releases, and A/B testing.
• Reduce inference latency through batching, quantization, and hardware-aware compilation.
• Monitor model quality, drift, and request throughput, and configure alerts.
• Allocate GPU and TPU capacity across training and serving, balancing performance with cost.

Requirements

• At least three years in ML infrastructure, MLOps, or backend systems engineering.
• Strong Kubernetes and Docker experience, plus cloud deployment work on GCP or AWS.
• Practical experience with a model-serving framework such as Triton, TorchServe, or TensorFlow Serving.
• Python proficiency and working knowledge of Go, Rust, or C++.
• Experience building ML delivery pipelines with tools such as MLflow or Kubeflow, or custom equivalents.
• Understanding of GPU inference optimization, including TensorRT or ONNX Runtime.

Nice to have

• Production experience with Vertex AI or SageMaker.
• Work on real-time services with strict latency targets, especially below 200 ms at p99.
• GPU workload cost estimation, distributed multi-GPU or multi-node training, or infrastructure-as-code with Terraform or Pulumi.