AI compute infrastructure engineer

Kraken; United States; Remote; USD 150,000 - 230,000; Full time

Job summary

Kraken ищет инженера инфраструктуры для команды AI Compute и Infrastructure. Вы будете управлять GPU-кучами и ускорителями, отвечать за драйверы, рантаймы и конфигурацию нод, а также за планирование ресурсов и мониторинг.

Ваша работа будет направлена на проектирование локального выполнения моделей, снижение зависимости от внешних провайдеров и оптимизацию расходов на вычисления, обеспечивая высокую доступность сервисов и быструю отдачу для исследователей и инженеров.

Qualifications

• 5+ лет опыта в инфраструктурной инженерии, GPU-вычислениях или крупных производственных платформах
• Опыт эксплуатации GPU-кластеров в продуктивной среде, включая планирование и мониторинг использования
• Сильные навыки системного программирования и отладки в Linux, сетях, контейнерах и Kubernetes
• Опыт работы с ML-серверами (vLLM, Triton, TensorRT и т.д.)
• Знание Python для автоматизации инфраструктуры и рабочих процессов
• Понимание компромиссов по производительности, памяти, задержке и стоимости

Responsibilities

• Управление и эксплуатация GPU-куч и ускорителей для обучения и вывода моделей
• Проектирование инфраструктуры для локального выполнения моделей
• Разработка планирования, оркестрации и квотирования в рамках гибридной среды
• Оптимизация конвейеров вывода для задержки, пропускной способности и затрат
• Сотрудничество с ML-инженерами для устранения узких мест
• Обеспечение наблюдаемости: метрики, логи, дашборды и инцидент-работы
• Повышение надежности, реагирование на инциденты и пост-фактум анализ
• Оценка и интеграция нового оборудования и сервисов
• Разработка инструментов управления использованием GPU и затратами

Skills

GPU compute

Linux

Kubernetes

Python

Performance optimization

Observability

Cost optimization

Distributed systems

ML serving frameworks

Tools

Kubernetes

vLLM

Triton Inference Server

TensorRT

TorchServe

KServe

Ray Serve

Cuda / driver debugging

Job description

Описание:

Kraken is a crypto platform offering spot trading, margin, futures, staking, and OTC services to individual investors and institutional clients. Its AI Compute and Infrastructure team builds infrastructure for model training, inference, evaluation, and experimentation across the exchange.

Задачи:

Own and operate GPU and accelerator clusters for training, inference, evaluation, and experimentation, including drivers, runtimes, kernels, device plugins, node configuration, scheduling primitives, and workload isolation. Design infrastructure that enables local GPU model execution where strategically and economically preferable, reducing dependence on external providers and controlling compute costs. Build and improve scheduling, orchestration, placement, quota management, and utilization systems across heterogeneous accelerator environments. Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost using vLLM, Triton Inference Server, TensorRT, or equivalent serving stacks. Partner with ML engineers and researchers to remove bottlenecks in training, evaluation, batch and online inference, deployment, and production debugging workflows. Build observability for GPU utilization, memory pressure, queue depth, saturation, token throughput, request latency, failed workloads, capacity pressure, and spend. Drive reliability, incident response, alerting, runbooks, and post-incident improvements for always-on AI compute infrastructure. Evaluate and integrate new hardware, cloud instance families, specialized accelerators, runtimes, schedulers, and serving frameworks. Build tooling that makes GPU usage visible, accountable, and easier for internal teams to use. Contribute to long-term architecture decisions balancing performance, cost efficiency, scalability, operational simplicity, and production safety.

Требования:

• 5+ Years of infrastructure engineering experience, including significant work in GPU compute, ML infrastructure, distributed systems, high-performance computing, or large-scale production platforms
• Hands-on experience operating GPU clusters or accelerator-backed infrastructure in production or production-like environments, including scheduling, orchestration, utilization monitoring, and cost optimization
• Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging
• Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve, or equivalent systems
• Proficiency in Python for infrastructure automation, tooling, debugging, integration, and operational workflows
• Practical understanding of performance tradeoffs across batching, concurrency, memory usage, GPU utilization, model size, latency, throughput, availability, and cost
• Track record of optimizing compute costs while maintaining clear performance, reliability, and availability expectations
• Experience building observable systems with useful metrics, logs, traces, dashboards, alerts, and incident workflows
• Comfortable working in high-stakes, always-on environments where uptime, throughput, correctness, and operational discipline are critical
• Clear communication skills and ability to explain infrastructure tradeoffs to researchers, product teams, platform engineers, security stakeholders, and engineering leadership
• Applicants may redact or remove resume information identifying age, date of birth, or dates of educational attendance or graduation
• Qualified applicants with criminal histories are considered in a manner consistent with the San Francisco Fair Chance Ordinance
• Будет плюсом: experience at a frontier AI lab, hyperscaler, high-frequency trading firm, research platform, or high-scale ML organization; familiarity with custom silicon or specialized accelerators such as TPUs, AWS Trainium, or Gaudi; background in capacity planning, procurement input, reserved capacity strategy, cloud accelerator economics, or GPU fleet cost management; experience with distributed training frameworks such as DeepSpeed, Megatron-LM, FSDP, or Ray; experience debugging CUDA, NCCL, kernel, driver, runtime, memory, networking, or low-level performance issues; experience with Rust, C++, Go, CUDA, or other systems languages used for performance-critical infrastructure; crypto, financial services, trading infrastructure, or security-sensitive production infrastructure experience

Условия:

Applications are accepted on an ongoing basis unless a specific application deadline is stated Job-related skills or work-style assessments may be part of the hiring process. They evaluate role-relevant competencies, are applied consistently across candidates for similar positions, and are considered alongside experience and interviews, not as the sole basis for an employment decision Equal opportunity employer; discrimination and harassment are not tolerated on protected grounds