Senior AI Inference Engineer - Hybrid Scale for LLMs

Reuters Agency; Eagan (MN); Hybrid; USD 110,000 - 204,000; Full time

Benefits offered by this job

Hybrid Work Model

Flexibility & Work-Life Balance

Career Development and Growth

Job summary

Thomson Reuters is seeking a Senior Inference Engineer, AI to collaborate with platform teams to productionize and scale AI and LLM workloads across AWS, Azure, GCP and internal Kubernetes clusters.

You will optimize models for low latency, develop containerized inference pipelines, implement routing and observability, and work with data science and product teams to onboard new research models into production.

Qualifications

• 5+ years of relevant experience.
• Strong understanding of ML/LLM fundamentals and inference optimization techniques.
• Hands-on experience with GPU programming (CUDA preferred), inference runtimes (TensorRT, ONNX Runtime), and deep learning frameworks (PyTorch/TensorFlow).
• Proficiency in Python and at least one systems language (C++ strongly preferred).
• Experience deploying AI workloads to AWS/GCP/Azure and Kubernetes.
• Familiarity with vector search systems (OpenSearch vectors) and retrieval augmented generation pipelines.
• Knowledge of distributed systems, microservices, CI/CD, and cloud native architecture.

Responsibilities

• Within Platform Engineering and Enterprise AI Services, an AI Inference Engineer is responsible for productionizing, optimizing, and scaling AI and LLM workloads.
• Deploy and scale inference workloads on GPUs across AWS, Azure, GCP and internal Kubernetes clusters.
• Implement routing and failover strategies for OpenAI/Anthropic/Vertex AI traffic.
• Collaborate with Platform Engineering teams to ensure inference workloads align with cloud-native patterns.
• Build and optimize containerized inference pipelines using Kubernetes for large-scale distributed workloads.
• Ensure compliance with AI standards for deployment, monitoring, governance, and drift detection.

Skills

GPU programming

CUDA

TensorRT

ONNX Runtime

PyTorch

Python

C++

AWS/GCP/Azure

Kubernetes

Vector search

RAG pipelines

Distributed systems

CI/CD

Tools

OpenSearch vectors

Job description

Thomson Reuters is seeking a Senior Inference Engineer, AI to collaborate with platform teams to productionize and scale AI and LLM workloads across AWS, Azure, GCP and internal Kubernetes clusters.

You will optimize models for low latency, develop containerized inference pipelines, implement routing and observability, and work with data science and product teams to onboard new research models into production.