ML Engineer

Base Compute; Melbourne; On-site; AUD 120,000 - 180,000; Full time

Benefits offered by this job

Equity

Competitive salary

Job summary

Base Compute, an AI inference lab focused on on-device intelligence, is hiring an ML Engineer to own and optimize the entire inference stack. You will drive performance across CPU/GPU, write production-ready systems in C++, and shape hardware- and software-level bets for edge platforms.

Applicants should have 3+ years in ML or systems programming, strong GPU know-how, and a track record of turning research into scalable infrastructure.

Qualifications

• 3+ years of experience in ML engineering or systems programming (Rust, C/C++).
• Expertise in GPU programming and hardware optimization across platforms (CUDA, ROCm, Metal).
• Solid understanding of modern LLM architectures and optimization techniques (quantization, decoding).
• Strong ownership and autonomy: translate research into production-ready infrastructure.
• Good communication: explain complex decisions and document systems clearly.

Responsibilities

• Develop and scale our on-device inference engine and runtime.
• Optimize kernels and leverage hardware features across architectures.
• Build low-latency serving runtimes in C++ for routing, batching, and decoding strategies.
• Profiling to remove bottlenecks from memory to kernel interleaving.

Skills

ML engineering

Systems programming

Rust

C/C++

GPU programming

Hardware optimization

Ownership

Communication

Tools

CUDA

ROCm

Metal

Triton

Torch.compile

Custom operators

Job description

About Us

Base Compute is an AI inference lab. Our mission is to bring AGI on device. We believe in a world where everyone has access to intelligence: fast, private and always available on your device.

We’re building the infrastructure for the next generation of on-device AI, from silicon-level optimizations to distributed inference systems.

We’re working on hard problems at the intersection of inference efficiency, model intelligence and autonomous research.

The Role

We’re looking for an ML Engineer to work at the frontier of on-device AI. This role is for someone who lives at the intersection of systems engineering and machine learning, turning state-of-the-art research into hyper-optimized, production-ready infrastructure.

You’ll have significant ownership over our entire inference stack and direct influence on the technical bets the company makes.

What You’ll Work On

• Inference engine development: Building and scaling our custom inference engine, handling everything from weight loading and KV-cache management to efficient request scheduling
• Cross-platform silicon optimization: Writing and tuning custom kernels and leveraging hardware-specific instructions to squeeze maximum performance out of diverse architectures, including Apple Silicon, NVIDIA, AMD, Snapdragon, and other edge platforms
• Systems architecture: Developing robust, low-latency serving runtimes in C++ to manage model routing, continuous batching, and novel decoding strategies under strict thermal and memory constraints
• Performance profiling: Identifying and eliminating bottlenecks across the entire stack, from memory bandwidth ceilings to kernel interleaving

What We’re Looking For

• 3+ years of experience in ML engineering or systems programming (Rust, C/C++), with a strong track record of building performance-critical software
• Expertise in GPU programming and hardware optimization across various platforms (CUDA, ROCm, Metal, Triton, or similar)
• Solid understanding of modern LLM architectures, including parsing formats and implementing optimization techniques (quantization, speculative decoding, etc.)
• A strong sense of ownership and autonomy: the ability to take ambiguous architectural challenges and drive them from research translation directly into production-ready infrastructure
• Good communication: the ability to explain complex architectural decisions simply, give honest feedback and document systems cleanly
• Nice-to-haves:
  • Familiarity with ML compilers (torch.compile, custom operators)
  • Experience with low-precision inference (INT8/FP8/FP4)
  • Knowledge of Edge LLMOps

What we offer

• Equity and strong base salary
• Direct influence on technical direction: your ideas will shape the roadmap
• Work on genuinely hard problems that haven't been solved yet
• Small team, fast iteration, low bureaucracy

Location

The team is based in Melbourne and Berlin and works in-person from the office most days. We require strong written and spoken English, since the team collaborates across time zones.