Senior Software Engineer: Performance ML Inference (Remote)

positron.ai; United States; Remote; USD 150,000 - 230,000; Full time

Benefits offered by this job

Remote-first policy

Comprehensive benefits (medical, etc.)

Unlimited paid time off

13 paid holidays

Equity

401(k) with matching

Job summary

positron.ai is seeking an experienced engineer to build and optimize software that runs open-source large language models on a specialized inference appliance combining x86 CPUs and FPGAs. The role focuses on modern C++ performance across the entire execution path, from vectorized kernels to transformer features such as token streaming and KV caching.

You will collaborate with FPGA and systems engineers to optimize data movement, improve transformer execution, and ensure robust CI testing.

Qualifications

• At least 7 years of professional C++ development focused on performance-critical software.
• Strong grasp of C++ templates and modern memory management.
• Practical SIMD experience, such as AVX-512 or SSE, including intrinsics-based vectorization.
• Experience with high-performance computing and parallel programming techniques.

Responsibilities

• Develop inference software and C++ libraries that use SIMD, threading, and the memory hierarchy efficiently.
• Coordinate with FPGA and systems engineers on data movement and computation offload between CPUs and accelerators.
• Improve model execution through vectorization, cache-aware techniques, hardware-aware scheduling, and NUMA-aware memory management.
• Profile instruction and data-flow bottlenecks and contribute tools and methods for performance analysis.
• Implement transformer execution improvements, including token streaming, KV cache management, and batching.
• Work with ML colleagues on quantization, sparsity, and mixed-precision execution; maintain unit, performance, acceptance, and regression tests in CI.

Skills

C++ performance

Templates

Modern memory management

Threading

Parallel programming

Tools

AVX-512

SSE

SIMD intrinsics

Job description

positron.ai is seeking an experienced engineer to build and optimize software that runs open-source large language models on a specialized inference appliance combining x86 CPUs and FPGAs. The role focuses on modern C++ performance across the entire execution path, from vectorized kernels to transformer features such as token streaming and KV caching.

You will collaborate with FPGA and systems engineers to optimize data movement, improve transformer execution, and ensure robust CI testing.