Remote AI Inference Platform Architect

Utilidata; United States; Remote; USD 170,000 - 210,000; Full time

Benefits offered by this job

Stock options

Health insurance

401k match

Flexible PTO

Job summary

Utilidata is hiring an AI Infrastructure Engineer to design, build, and own end-to-end infrastructure for AI/ML models across edge, cloud, and data centers.

This role emphasizes reliable, high-performance GPU workloads with low latency and cost efficiency, collaborating with product, engineering, and data science teams. Remote-friendly with periodic on-site engagements and company retreats.

Qualifications

• 5+ years of software engineering with a strong AI infrastructure focus.
• Hands-on with AI model serving frameworks (e.g., vLLM, Triton, TorchServe).
• Experience with Kubernetes and Docker in production.
• Deploying and operating infra across data center and on-prem environments.
• Strong GPU workload knowledge and inferences vs training.
• Proficiency in Python; C++, CUDA; Go and Rust are a plus.
• Excellent cross-functional communication in fast-moving teams.
• Willingness to travel up to 10% of time.

Responsibilities

• Lead design and build of AI inference platform, establishing scalable patterns and operational practices.
• Own end-to-end model serving infrastructure for on-prem and datacenter deployments.
• Build fault-tolerant, high-performance systems for AI models with low latency and cost efficiency.
• Collaborate with algorithms engineers to align inference data and power optimization.
• Optimize GPU utilization across hardware fleet, including NVIDIA accelerators.
• Establish MLOps practices with CI/CD, monitoring, and rollback across environments.
• Contribute to infrastructure roadmap, evaluating build vs buy and tooling evolution.

Skills

AI infrastructure

Model serving

Kubernetes

Docker

Python

C++

CUDA

Go

Rust

Communication

Travel

Tools

Kubernetes

Docker

Terraform

Helm

Job description

Utilidata is hiring an AI Infrastructure Engineer to design, build, and own end-to-end infrastructure for AI/ML models across edge, cloud, and data centers.

This role emphasizes reliable, high-performance GPU workloads with low latency and cost efficiency, collaborating with product, engineering, and data science teams. Remote-friendly with periodic on-site engagements and company retreats.