Performance & Debug Software Engineer – AI Inference

positron.ai; United States; Remote; USD 160,000 - 260,000; Full time

Benefits offered by this job

Medical coverage

Dental coverage

Vision coverage

Life insurance

Disability insurance

Equity

401(k) match

Unlimited PTO

13 paid holidays

Job summary

positron.ai is seeking an experienced engineer to improve end-to-end performance for large language model inference on custom accelerator systems. You will work across attention, data movement, and serving layers to optimize latency and throughput.

You will profile using counters and traces, develop distributed inference features including KV-cache management, and build benchmarks to expose limits. The role is remote-first in the United States with equity and comprehensive benefits.

Qualifications

• Strong C++ and Python skills with Linux systems profiling and optimization.
• Experience profiling and optimizing complex Linux systems that use VFIO interfaces.
• Hands-on profiling experience; tools and further qualifications not listed in the ad.

Responsibilities

• Profile and optimize latency, throughput, utilization, and efficiency across accelerator, runtime, host, and on-device execution.
• Use performance counters, traces, telemetry, and simulation to isolate regressions and assess fixes.
• Develop distributed inference capabilities, including KV-cache management and data movement across heterogeneous hardware.
• Build representative benchmarks and workload traces that expose compute, memory, interconnect, networking, scheduling, and model-loading limits.
• Turn performance findings into regression tests and release gates in collaboration with test engineers.
• Inform hardware and software architecture where measurements reveal structural performance constraints.

Skills

C++

Python

Profiling

VFIO

Linux

Job description

Role overview

Improve end-to-end performance for large language model inference on custom accelerator systems. The work spans attention computation, accelerator configuration, distributed KV-cache management, host-to-device data movement, runtime behavior, and serving systems. You will use real workloads and measured evidence to identify bottlenecks, implement improvements, and help shape hardware and software interfaces.

Responsibilities

• Profile and optimize latency, throughput, utilization, and efficiency across accelerator, runtime, host, and on-device execution.
• Use performance counters, traces, telemetry, and simulation to isolate regressions and assess fixes.
• Develop distributed inference capabilities, including KV-cache management and data movement across heterogeneous hardware.
• Build representative benchmarks and workload traces that expose compute, memory, interconnect, networking, scheduling, and model-loading limits.
• Turn performance findings into regression tests and release gates in collaboration with test engineers.
• Inform hardware and software architecture where measurements reveal structural performance constraints.

Requirements

• Strong C++ and Python skills, with practical experience profiling and optimizing complex Linux systems that use VFIO interfaces.
• Deep understanding of memory hierarchies, concurrency, data movement, batching, scheduling, and performance measurement.
• Hands-on profiling experience; the available source cuts off before listing the specific tools or any further qualifications.

Benefits and work setup

The role is remote-first, with a company-provided computer and home-office setup.

• company-paid medical, dental, and vision coverage
• life and disability coverage
• supplemental insurance options
• unlimited paid time off
• 13 paid holidays
• equity
• a 401(k) match

Applicants must already be authorized to work in the U.S.; new visa sponsorship is unavailable, though eligible H-1B transfers may be supported.