Staff Inference Engineer — Local AI & Performance

Ghost; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Ghost, a personal AI hardware company in San Francisco, is hiring an engineer to build and optimize the inference stack that runs personal AI models locally on Core.

You will improve latency, throughput and memory efficiency across interactive requests and background workloads, profile across the stack, and collaborate with hardware, post-training and product teams.

The role requires strong C++ and Python skills, familiarity with GPU architecture and CUDA/Triton, and on-site work in San Francisco.

Qualifications

• Experience building or optimizing inference systems for language models.
• Strong systems programming in C++ and proficient Python.
• Familiarity with GPU memory management and performance profiling.

Responsibilities

• Build and optimize the inference stack that runs personal AI models locally.
• Improve latency, throughput, and memory efficiency across workloads.
• Profile performance across hardware and software stacks, optimizing kernels and memory movement.
• Build benchmarks reflecting real product workloads with long contexts and concurrent agents.
• Collaborate with hardware, post-training, and product teams.

Skills

Inference systems

C++ programming

Python

GPU architecture

Performance profiling

Experimental judgment

Tools

CUDA

Triton

Job description

Ghost, a personal AI hardware company in San Francisco, is hiring an engineer to build and optimize the inference stack that runs personal AI models locally on Core.

You will improve latency, throughput and memory efficiency across interactive requests and background workloads, profile across the stack, and collaborate with hardware, post-training and product teams.

The role requires strong C++ and Python skills, familiarity with GPU architecture and CUDA/Triton, and on-site work in San Francisco.