Performance Engineer

Omniva; Bellevue (WA); On-site; USD 200,000 - 300,000; Full time

Job summary

Omniva in Bellevue is hiring a founding performance engineer for Token Factory, a serverless inference platform. You’ll embed with customers, own deployments end to end, and demonstrate gains in tokens per second and time to value during early engagements.

You’ll tune the serving stack (vLLM, SGLang, TensorRT-LLM), optimize models and hardware across NV/AMD fleets, and collaborate with labs. You’ll publish benchmarks and write client-facing engagement reports.

Qualifications

• Senior or Staff level requires 4+ years in performance engineering, ML systems or HPC.

Responsibilities

• Deliver measurable gains for customers from named deployments and tuned production configs.
• Tune the serving system across vLLM, SGLang and TensorRT-LLM on diverse fleets.
• Optimize model and algorithm with FP8/NVFP4/MXFP4 quantization and speculative decoding.
• Tune architecture and hardware with MoE, kernels and advanced attention backends.
• Lead performance agents, review proposals and feed data back to the system.
• Engage with labs and partners to run open-weight releases optimized Day 0.
• Publish reproducible benchmarks and upstream contributions; write engagement reports.

Skills

LLM serving stack

GPU performance fundamentals

LLM inference internals

Customer interaction

Automation

Clear writing

Job description

About The Role

Omniva is building Token Factory, a serverless and dedicated inference platform for the best open-weight models, running on infrastructure we own. We're hiring a founding team of ten performance engineers who work directly with customers: AI-native startups, agent platforms and frontier AI labs. Your job is to take a customer's model from "it runs" to "this is the fastest deployment of this model anywhere", and to show that gain in the customer's first weeks on the platform. You'll embed with customers and own each deployment end to end. You're measured on what you deliver: tokens per second per user, time to first token, cost per million tokens and time to value. You'll work at every layer that sets speed: the serving system, the model, the algorithm and the hardware. Our in-house performance agents already automate a growing share of the tuning loop. You'll direct them, check their work and train them with what you learn on each engagement.

About The Company

Omniva

Responsibilities

• Deliver measurable gains for customers. Own named deployments on Token Factory dedicated endpoints, from the intake benchmark to a tuned production config. Show a measured gain within the first two to four weeks of each engagement.
• Tune the serving system. Profile and tune vLLM, SGLang and TensorRT-LLM: parallelism strategy across TP, PP, EP and DP, batching and scheduling, prefix caching, KV-cache layout, prefill and decode disaggregation, and attention backend selection. You'll do this on NVIDIA Blackwell B200, B300 and GB300 fleets and on AMD MI355X fleets.
• Optimize the model and the algorithm. Apply and validate FP8, NVFP4 and MXFP4 quantization, speculative decoding such as EAGLE, suffix and SMC-style variants, distillation-aware serving and KV-precision strategies. Every change passes quality regression gates, so a speed gain never costs accuracy without anyone seeing it.
• Optimize for the architecture and the hardware. For fine-grained MoE and long-context models, choose and tune expert-parallel layouts, grouped-GEMM and MoE kernels, and communication patterns. Work with FlashAttention, FlashInfer, Liger, CUTLASS and ThunderKittens, and write custom Triton or CUDA kernels when the gain justifies it.
• Run and supervise our performance agents. Point them at customer workloads, review and approve what they propose, and feed every engagement back into the system. Your judgment becomes the training data that makes the next engagement faster.
• Work with labs and partners. Engage early with frontier labs and model providers so that new open-weight releases run optimized on Token Factory from day 0. Work with technology partners on kernels, speculative decoding and serving engine improvements.
• Publish your work. Contribute reproducible benchmarks, best-known configurations and upstream PRs to the serving ecosystem. Represent Omniva's performance work in blog posts, on leaderboards such as InferenceMAX and Artificial Analysis, and in communities such as GPU MODE.
• Report like an owner. Write engagement reports for customers and internal teams that say what you tried, what moved the metric, where the ceiling is and what you'd do next. Explain each engineering gain in terms the customer's leadership cares about.

Qualifications

• 4+ years for Senior, or 7+ years for Staff, in performance engineering, ML systems, HPC or inference infrastructure. Equivalent depth shown through open-source contributions, leaderboard results or published kernels counts too.

Required Skills

• Production experience with at least one major LLM serving stack: vLLM, SGLang or TensorRT-LLM. You've shipped and tuned real deployments, well past the quickstart.
• Strong GPU performance fundamentals: roofline analysis, memory-bound versus compute-bound reasoning, and profiling with Nsight, DCGM, Kineto or similar tools. You benchmark with statistical rigor and control for variance, warmup and clock effects.
• Working knowledge of LLM inference internals: attention variants and KV caching, continuous batching, quantization formats and their quality trade-offs, speculative decoding, and MoE routing and expert parallelism.
• You're good with customers. You can sit with a customer's engineers, earn their trust in the first session, push back when their ask is wrong and own the outcome.
• You're practical about automation. You direct agentic tools, audit their output and overrule them when they're wrong.
• You write clearly. Much of this job is configs, benchmark reports and recommendations that other people act on.

Preferred Skills

• CUDA, Triton or HIP kernel development, or contributions to FlashAttention, FlashInfer, Liger, CUTLASS, DeepGEMM or similar.
• Experience with AMD ROCm, or with non-GPU accelerators such as TPU, Trainium, wafer-scale or dataflow chips.
• Tuning for agentic workloads: coding agents, traffic with high prefix reuse, multi-turn sessions with many tool calls, and tokens per second per user as the main metric.
• Forward-deployed, solutions architect or consulting experience at an infrastructure or ML platform company.
• Contributions to open benchmarks or competitions such as KernelBench, GPU MODE KernelBot, MLPerf Inference or InferenceMAX.
• Experience applying RL, Bayesian optimization, evolutionary search or agentic code generation to systems problems.

Pay

The base salary for this role is $200,000 to $300,000 a year, depending on level and experience. Total compensation also includes an annual performance bonus, a long-term incentive and benefits.

Omniva is an equal opportunity employer. We consider all qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status or any other characteristic protected by law. If you need an accommodation at any point in the hiring process, let us know.