Member of Technical Staff - Inference

Ghost; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Ghost, a personal AI hardware company in San Francisco, is hiring an engineer to build and optimize the inference stack that runs personal AI models locally on Core.

You will improve latency, throughput and memory efficiency across interactive requests and background workloads, profile across the stack, and collaborate with hardware, post-training and product teams.

The role requires strong C++ and Python skills, familiarity with GPU architecture and CUDA/Triton, and on-site work in San Francisco.

Qualifications

• Experience building or optimizing inference systems for language models.
• Strong systems programming in C++ and proficient Python.
• Familiarity with GPU memory management and performance profiling.

Responsibilities

• Build and optimize the inference stack that runs personal AI models locally.
• Improve latency, throughput, and memory efficiency across workloads.
• Profile performance across hardware and software stacks, optimizing kernels and memory movement.
• Build benchmarks reflecting real product workloads with long contexts and concurrent agents.
• Collaborate with hardware, post-training, and product teams.

Skills

Inference systems

C++ programming

Python

GPU architecture

Performance profiling

Experimental judgment

Tools

CUDA

Triton

Job description

About Ghost

Ghost is a personal AI hardware company.

To realize the full benefit of what models will do for people, we believe an entirely new platform will be built around AI. Some devices will let your AI understand what is happening around you, others will enable you to get things done without navigating apps or menus.

As these systems become more capable and more personal, it matters who owns them. We’re building a future where that intelligence belongs to you, with an ecosystem of devices and software that gives you greater control over your time, attention, and the things you want to accomplish.

What you'll do

• Build and optimize the inference stack that runs personal AI models locally on Core.

• Improve latency, throughput, and memory efficiency across interactive requests and background agent workloads.

• Profile performance across the hardware and software stack, optimizing kernels, memory movement, and runtime execution.

• Build benchmarks that reflect real product workloads, including long contexts, concurrent agents, and sustained operation.

• Work closely with the hardware, post-training, and product teams.

Who you are

• Experience building or optimizing inference systems for language models.

• Strong systems programming skills in C++ or similar languages, alongside proficiency in Python.

• Familiarity with GPU architecture, memory management, and performance profiling.

• Experience with inference runtimes, quantization, or GPU kernel development using CUDA, Triton, or equivalent tools.

• Strong experimental judgment: you can identify bottlenecks and evaluate tradeoffs between speed, memory, power, and model quality.

• Ability to work on-site in San Francisco.