Founding Inference Engineer, Apple Silicon Platform

Mount Thor; San Francisco (CA); On-site; USD 240,000 - 320,000; Full time

Job summary

Mount Thor is hiring a founding inference engineer to build a production inference platform on Apple Silicon. You will own the stack from GPU kernels and model execution to distributed scheduling, networking, and customer-facing serving systems.

You will define the architecture, implement performance-critical code, and shepherd the system from deployment to production operation, ensuring low latency, high throughput, and reliable service for customers.

Qualifications

• Production inference engines experience and ownership of performance-critical code.
• Strong systems programming skills in C++ or Rust with Python proficiency.
• Practical understanding of transformer inference, attention, and quantization.
• Experience with GPU programming, memory hierarchies, and performance analysis.
• Strong distributed-systems fundamentals including scheduling and networking.
• Experience bringing software from architecture to production deployment.

Responsibilities

• Build and optimize the inference stack from kernels to serving APIs.
• Design for Apple Silicon, profiling on real hardware and optimizing memory usage.
• Build distributed inference and its communication layer across machines.
• Own inference scheduling, routing, and autoscaling to meet latency targets.
• Ship a production inference service with observability and safe releases.

Skills

C++

Rust

Python

Systems programming

Distributed systems

Tools

MLX

llama.cpp

vLLM

TensorRT-LLM

Job description

Mount Thor is hiring a founding inference engineer to build a production inference platform on Apple Silicon. You will own the stack from GPU kernels and model execution to distributed scheduling, networking, and customer-facing serving systems.

You will define the architecture, implement performance-critical code, and shepherd the system from deployment to production operation, ensuring low latency, high throughput, and reliable service for customers.