Senior AI Compute Infrastructure Engineer

Favia; United States; Remote; USD 180,000 - 300,000; Full time

Job summary

Favia is building a dedicated AI Compute and Infrastructure team to power the next generation of model training, inference, evaluation, and experimentation across the exchange. This team sits within engineering leadership and owns the infrastructure layer that lets Favia run AI workloads with control, speed, reliability, and cost discipline.

You will join a small, senior, high-impact team working directly with AI/ML researchers, platform engineers, security teams, and product teams.

Qualifications

• 5+ years of infrastructure engineering experience with GPU compute or ML infra.
• Hands-on experience operating GPU clusters in production, including scheduling and cost optimization.
• Strong Linux, networking, storage, containers, and Kubernetes fundamentals.
• Experience with ML serving frameworks such as vLLM, Triton, TensorRT, TorchServe, KServe, Ray Serve.
• Proficiency in Python for automation, tooling, and pipelines.
• Understanding of performance tradeoffs across batching, memory usage, latency, throughput, and cost.
• Experience building observable systems with metrics, logs, traces, dashboards, and incident workflows.
• Clear communication of infrastructure tradeoffs to researchers, product teams, security, and leadership.

Responsibilities

• Own and operate GPU and accelerator clusters for training, inference, evaluation, and experimentation.
• Design infrastructure enabling local model runs on GPUs to reduce dependency on external providers and control costs.
• Build and improve scheduling, orchestration, placement, quota management, and utilization across heterogeneous accelerators.
• Optimize inference pipelines for latency and throughput using ML serving stacks.
• Partner with ML engineers to remove bottlenecks in training, evaluation, and production workflows.
• Build observability for GPU utilization, memory pressure, queue depth, latency, and spend.
• Drive reliability, incident response, runbooks, and post-incident improvements for always-on AI compute.
• Evaluate new hardware, accelerators, and serving frameworks as the landscape evolves.
• Develop tooling to make GPU usage visible and consumable by internal teams.
• Contribute to long-term architecture decisions balancing performance, cost, and scalability.

Skills

GPU compute infra

Distributed systems

Linux fundamentals

Python automation

Observability

Performance tradeoffs

Interdisciplinary collaboration

Tools

Kubernetes

Docker

CUDA

TensorRT

NCCL

TorchServe

Job description

The Team

Favia is building a dedicated AI Compute and Infrastructure team to power the next generation of model training, inference, evaluation, and experimentation across the exchange. This team sits within engineering leadership and owns the infrastructure layer that lets Favia run AI workloads with control, speed, reliability, and cost discipline.

The team is responsible for GPU and accelerator infrastructure, cluster operations, scheduling, model serving, observability, capacity planning, and cost-efficient compute at scale. This is the backbone that allows Favia to train, serve, evaluate, and iterate on AI systems in-house where it matters for privacy, latency, reliability, cost, or product differentiation.

You will join a small, senior, high-impact team working directly with AI/ML researchers, platform engineers, security teams, and product teams. The mandate is simple: make Favia's AI ambitions real by building compute infrastructure that is fast, dependable, efficient, and production-grade.

The opportunity

• Own and operate GPU and accelerator clusters used for training, inference, evaluation, and experimentation, including drivers, runtimes, kernels, device plugins, node configuration, scheduling primitives, and workload isolation.

• Design infrastructure that enables Favia teams to run models locally on GPUs where it is strategically and economically preferable, reducing unnecessary dependency on external providers and containing compute costs.

• Build and improve scheduling, orchestration, placement, quota management, and utilization systems across heterogeneous accelerator environments.

• Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost using frameworks such as vLLM, Triton Inference Server, TensorRT, or equivalent serving stacks.

• Partner with ML engineers and researchers to remove bottlenecks in training, evaluation, batch inference, online inference, deployment, and production debugging workflows.

• Build observability for GPU utilization, memory pressure, queue depth, saturation, token throughput, request latency, failed workloads, capacity pressure, and spend.

• Drive reliability, incident response, alerting, runbooks, and post-incident improvements for always-on AI compute infrastructure.

• Evaluate and integrate new hardware, cloud instance families, specialized accelerators, runtimes, schedulers, and serving frameworks as the AI infrastructure landscape evolves.

• Build tooling that makes GPU usage visible, accountable, and easier for internal teams to consume without needing to become infrastructure experts.

• Contribute to long-term architecture decisions that balance performance, cost efficiency, scalability, operational simplicity, and production safety.

What You Bring

• 5+ years of infrastructure engineering experience, with significant time spent on GPU compute, ML infrastructure, distributed systems, high-performance computing, or large-scale production platforms.

• Hands-on experience operating GPU clusters or accelerator-backed infrastructure in production or production-like environments, including scheduling, orchestration, utilization monitoring, and cost optimization.

• Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging.

• Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve, or equivalent systems.

• Proficiency in Python for infrastructure automation, tooling, debugging, integration, and operational workflows.

• Practical understanding of performance tradeoffs across batching, concurrency, memory usage, GPU utilization, model size, latency, throughput, availability, and cost.

• Track record of optimizing compute costs while maintaining clear performance, reliability, and availability expectations.

• Experience building observable systems with useful metrics, logs, traces, dashboards, alerts, and incident workflows.

• Comfortable working in high-stakes, always-on environments where uptime, throughput, correctness, and operational discipline are critical.

• Clear communicator who can translate infrastructure tradeoffs for researchers, product teams, platform engineers, security stakeholders, and engineering leadership.

Nice to haves

• Experience at a frontier AI lab, hyperscaler, high-frequency trading firm, research platform, or high-scale ML organization.

• Familiarity with custom silicon or specialized accelerators such as TPUs, AWS Trainium, Gaudi, or similar platforms.

• Background in capacity planning, procurement input, reserved capacity strategy, cloud accelerator economics, or GPU fleet cost management.

• Experience with distributed training frameworks such as DeepSpeed, Megatron-LM, FSDP, Ray, or equivalent systems.

• Experience debugging CUDA, NCCL, kernel, driver, runtime, memory, networking, or low-level performance issues.

• Experience with Rust, C++, Go, CUDA, or other systems languages used for performance-critical infrastructure.

• Crypto, financial services, trading infrastructure, or security-sensitive production infrastructure experience.

Unless a specific application deadline is stated in the job posting, applications are accepted on an ongoing basis.

Please note, applicants are permitted to redact or remove information on their resume that identifies age, date of birth, or dates of attendance at or graduation from an educational institution.

We consider qualified applicants with criminal histories for employment on our team, assessing candidates in a manner consistent with the requirements of the San Francisco Fair Chance Ordinance.

Our commitment

We may ask candidates to complete job-related skills or work-style assessments as part of our hiring process. These assessments evaluate competencies relevant to the role and are applied consistently across candidates for similar positions. Results are considered alongside experience and interviews, and are not the sole basis for any employment decision.

As an equal opportunity employer, we don't tolerate discrimination or harassment of any kind, whether based on race, ethnicity, age, gender identity, citizenship, religion, sexual orientation, disability, pregnancy, veteran status, or any other protected characteristic as outlined by federal, state, or local laws.