GPU Systems Engineer

Career Techniques; New York (NY); Hybrid; USD 200,000 - 300,000; Full time

Job summary

Career Techniques in New York seeks an experienced infrastructure engineer to design, deploy, and scale large-scale GPU clusters for AI research. You will work across compute, storage, OS, and automation to support hundreds of petabytes and thousands of nodes.

You will profile GPU workloads, remove bottlenecks, and collaborate with researchers to translate findings into speedups. Expect to own end-to-end infrastructure projects from design through long-term support and vendor engagement.

Qualifications

• 5+ years engineering large-scale Linux systems in HPC/AI or distributed infra.
• Strong Linux fundamentals, including kernel-level troubleshooting.
• Hands-on debugging of distributed GPU workloads with a strong mental model of GPU performance.
• Experience with GPUDirect RDMA and data movement between GPUs and the network.
• Python for automation; CUDA or C/C++ for reading, profiling, and debugging GPU code.
• Familiarity with configuration tools such as Salt, Ansible, Puppet, or Chef.
• Ability to diagnose cross-stack issues across hardware, OS, and network.
• Clear communication with researchers, engineers, and vendors.

Responsibilities

• Design, deploy, and scale distributed GPU clusters from hardware selection to production operation.
• Profile GPU workloads and identify performance bottlenecks across compute, storage, and network.
• Collaborate with researchers to benchmark workloads and implement speedups.
• Build automation to operate thousands of nodes, including provisioning and self-healing.
• Own infrastructure projects end-to-end from scope to long-term support.
• Engage with vendors to root-cause complex hardware and software issues.

Skills

Linux systems

GPU workloads

Python

CUDA/C++

GPUDirect RDMA

Diagnostics

Communication

Tools

Salt

Ansible

Puppet

Chef

Job description

As part of R&D, you will join the engineers responsible for the compute, storage, operating systems, and automation behind that work at serious scale: hundreds of petabytes of storage and large CPU and GPU clusters spanning thousands of nodes. The role is broad by design. One week you might be shaping the architecture of a new AI cluster, the next profiling a training job that will not scale, the next writing automation that keeps the whole fleet healthy with minimal human intervention.

Responsibilities:

• Design, deploy, and scale distributed GPU clusters, from hardware selection and network topology through to production operation.
• Track down performance bottlenecks across the full stack: compute, storage, network, and the seams between them.
• Partner with researchers to profile and benchmark GPU workloads, then turn the findings into measurable speedups.
• Build the automation that lets a small team operate thousands of nodes: provisioning, monitoring, diagnostics, and self-healing.
• Own infrastructure projects end to end, from scope and design through implementation and long-term support.
• Qualify new generations of hardware and software, and work directly with vendors to root-cause complex issues.

Qualifications:

• 5+ years engineering large-scale Linux systems in HPC, AI, or distributed-infrastructure environments.
• Deep Linux fundamentals: installation, performance tuning, and debugging, down to the kernel when the problem calls for it.
• Hands-on troubleshooting of distributed GPU workloads, with a strong mental model of GPU performance.
• Working experience with GPUDirect RDMA. You understand how data moves between GPUs and the network, and what to check when it does not.
• Solid Python for automation and tooling, plus CUDA or C/C++ experience. You can read, profile, and debug GPU code, not just operate the clusters it runs on.
• Familiarity with configuration management tools such as Salt, Ansible, Puppet, or Chef.
• Comfort diagnosing problems that cross hardware, OS, and network boundaries rather than stopping at one layer.
• Clear communication. You will work daily with researchers, engineers, and vendors.

Comp: 200-300K + Bonus