Software Engineer, AI Infrastructure – LVM Inference & Evaluation

Jobtailor

Redwood City (CA)

On-site

USD 180,000 - 240,000

Full time

4 days ago

Be an early applicant

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

Jobtailor is building scalable AI infrastructure to support real‑time computer vision and multimodal inference, from data collection to deployment. The platform focuses on efficient model serving, benchmarking, and continuous improvement across large video and sensor data streams.

The team collaborates with researchers, product engineers, and infrastructure teams to optimize batching, caching, quantization, and GPU utilization for production readiness.

Qualifications

2+ years of industry experience building infrastructure, distributed systems, machine learning platforms, or production AI systems.
BS/MS in Computer Science or a related technical field, or equivalent practical experience.
Strong programming background, especially in Python, with solid software engineering fundamentals.
Experience designing and building scalable machine learning infrastructure for training, inference, evaluation, and deployment.
Hands‑on experience running deep learning models in production, ideally including LLMs, LVMs, vision‑language models, or multimodal models.
Strong understanding of inference optimization, including batching, caching, quantization, parallelism, memory optimization, GPU utilization, and latency reduction.
Experience with model-serving frameworks such as vLLM, Triton Inference Server, or similar technologies.
Experience building evaluation frameworks, test harnesses, benchmarks, regression tests, or model‑quality measurement systems.
Strong background in machine learning and deep learning.
Experience designing data engines or pipelines for training and evaluation data.
Familiarity with LLMs, LVMs, RAG pipelines, embedding models, or multimodal models in production applications.
Experience with cloud infrastructure, containers, orchestration, distributed systems, and GPU‑based workloads.
Nice to have: CUDA, NCCL, PyTorch, TensorRT, ONNX, video understanding, model compression, speculative decoding, distillation, pruning, prompt evaluation, vector databases, re‑rankers, search infrastructure, or internal ML platforms.
Strong collaboration and communication skills.
Proactive problem‑solving ability, ownership mindset, and adaptability.

Responsibilities

Design, build, and maintain AI infrastructure for real‑time computer vision, LLM, LVM, and multimodal workloads.
Develop evaluation harnesses and benchmarking systems for model quality and system performance.
Collaborate with researchers and engineers to productionize advances and optimize serving architectures.
Improve data engines, observability, monitoring, and debugging tools for production AI systems.

Skills

Python Programming

Distributed Systems

ML Infrastructure

Model Serving

GPU Utilization

Cloud Infrastructure

CUDA

PyTorch

TensorRT

ONNX

Containers

Orchestration

Distributed Systems

Video Understanding

Education

BS/MS in Computer Science or related field

Tools

VLLM

Triton Inference Server

CUDA

PyTorch

TensorRT

ONNX

Containers

Orchestration

Job description

Design, build, and maintain AI infrastructure for real‑time computer vision, LLM, LVM, and multimodal inference workloads. Build scalable systems for state‑of‑the‑art models across large volumes of video and sensor data. Optimize inference for latency, throughput, GPU utilization, reliability, and cost. Develop evaluation harnesses and benchmarking systems for model quality, system performance, regressions, and production readiness. Build infrastructure for continuous model evaluation, experimentation, and deployment. Partner with research scientists to productionize advances in computer vision, LLMs, LVMs, RAG, and multimodal AI. Improve model-serving architecture through batching, caching, routing, quantization, model parallelism, and hardware utilization. Develop data engines and feedback loops for training data collection, model behavior evaluation, and continuous AI improvement. Create observability, monitoring, and debugging tools for production AI systems. Define best practices for deploying, evaluating, and operating AI systems in enterprise environments. Collaborate with research scientists, product engineering, infrastructure teams, and stakeholders.

Requirements

2+ years of industry experience building infrastructure, distributed systems, machine learning platforms, or production AI systems
BS/MS in Computer Science or a related technical field, or equivalent practical experience
Strong programming background, especially in Python, with solid software engineering fundamentals
Experience designing and building scalable machine learning infrastructure for training, inference, evaluation, and deployment
Hands‑on experience running deep learning models in production, ideally including LLMs, LVMs, vision‑language models, or multimodal models
Strong understanding of inference optimization, including batching, caching, quantization, parallelism, memory optimization, GPU utilization, and latency reduction
Experience with model-serving frameworks such as vLLM, Triton Inference Server, or similar technologies
Experience building evaluation frameworks, test harnesses, benchmarks, regression tests, or model‑quality measurement systems
Strong background in machine learning and deep learning
Experience designing data engines or pipelines for training and evaluation data
Familiarity with LLMs, LVMs, RAG pipelines, embedding models, or multimodal models in production applications
Experience with cloud infrastructure, containers, orchestration, distributed systems, and GPU‑based workloads
Nice to have: large‑scale GPU infrastructure, CUDA, NCCL, PyTorch, TensorRT, ONNX, video understanding, model compression, speculative decoding, distillation, pruning, prompt evaluation, vector databases, re‑rankers, search infrastructure, or internal ML platforms
Strong collaboration and communication skills
Proactive problem‑solving ability, ownership mindset, and adaptability

Core Competencies

Demonstrates expertise in building and optimizing AI infrastructure for real‑time computer vision and multimodal inference workloads, with a strong focus on model evaluation, deployment, and performance optimization. Proficient in collaborating with cross‑functional teams to enhance model‑serving architectures and ensure production readiness.

Highest‑signal resume keywords