MLOps Engineer: Production GPU Benchmark & Validation

Wall Street Consulting Services LLC; Plano (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Wall Street Consulting Services LLC is seeking a senior ML engineering professional to drive workload engineering and validation initiatives for GPU-based systems. You will optimize distributed training across multi-node clusters and architect high-performance LLM inference serving with Triton, vLLM, and TensorRT-LLM.

Strong Python skills and experience with Kubernetes-native workflows are required. You will deliver rigorous benchmarks, validation reports, and PoC designs that close engagements.

Qualifications

• 5+ years ML engineering, MLOps, or performance engineering with production GPU workloads.
• Hands-on distributed training on multi-node GPU clusters — not single-GPU or notebook-scale work.
• Production LLM inference serving: Triton, vLLM, or TensorRT-LLM.
• Rigorous benchmarking discipline — profile a system, explain where GPU time goes, defend a number.
• Strong Python; containers and Kubernetes-native workflows (Kubeflow, Ray, Argo Workflows).
• Clear technical writing. A validation report is a deliverable a customer pays for.

Responsibilities

• Enable and tune distributed training: PyTorch DDP/FSDP, DeepSpeed/ZeRO, tensor/pipeline parallelism, NCCL tuning, MFU measurement.
• Architect inference serving with Triton, NVIDIA NIM, vLLM, TensorRT-LLM; quantization, batching, KV-cache, paged attention, autoscaling to latency SLOs.
• Size inference platforms backward from customer SLOs: time-to-first-token, inter-token latency, concurrency, context length.
• Support GenAI patterns: RAG, fine-tuning, agentic pipelines, vector database integration.
• Build and own GPU infra validation suite: fabric validation, NCCL curves, GPU burn/thermal, storage throughput, GPUDirect verification.
• Deliver acceptance validation engagements and author signed acceptance reports.
• Design the continuous-validation service: post-downtime revalidation, driver/firmware matrix certification, drift detection.
• Publish benchmark methodology/reports to establish credibility.
• Own PoC workload design and benchmark evidence to close engagements.

Skills

Distributed training

LLM inference

Python

Kubernetes workflows

Technical writing

Tools

Triton

vLLM

TensorRT-LLM

Job description

Wall Street Consulting Services LLC is seeking a senior ML engineering professional to drive workload engineering and validation initiatives for GPU-based systems. You will optimize distributed training across multi-node clusters and architect high-performance LLM inference serving with Triton, vLLM, and TensorRT-LLM.

Strong Python skills and experience with Kubernetes-native workflows are required. You will deliver rigorous benchmarks, validation reports, and PoC designs that close engagements.