Role & Responsibilities
- Speculative decoding. Train, tune and ship draft models for the models we run block-diffusion drafters (DFlash2-like), semi-autoregressive drafters with confidence-scheduled verification (DSpark), EAGLE-style heads and multi-token prediction and keep pushing acceptance length up and drafting cost down on consumer hardware.
- Kernels. Write, fuse and tune kernels for attention, quantised matmul, MoE routing, sampling and draft verification, across Metal/MLX, CUDA, Vulkan and OpenVINO.
- Quantisation. Design mixed-precision schemes for weights, activations and KV cache. Match formats to the hardware, from 4-bit integer on integrated GPUs and NPUs to FP8, FP4 and microscaling formats on newer GPUs, and prove what each one costs in quality before it ships.
- Prefill and decode. Push time-to-first-token and tokens per second with flash and paged attention, efficient handling of sliding-window and hybrid attention, chunked prefill and kernel-level scheduling.
- Tiered memory. Build hierarchical KV and context caching that spans GPU memory, system RAM and SSD, with prefix reuse, smart eviction and prefetching that hides I/O, so long contexts and multiple models fit on machines that shouldn't be able to hold them.
- Making it work everywhere. Build the test matrix, performance telemetry and regression gates that catch a slow driver, a wrong numerical result or a stalled UI before users do.
- Instrumentation. Expose hooks into the runtime, such as activations and attention, so research and security work can see inside models without giving up the speed of fused kernels.
Job Requirements
- Deep, hands-on GPU programming in at least one of CUDA, Metal/MLX or Vulkan compute, plus evidence that you've carried that work over to a second backend.
- Hands-on speculative decoding experience: you've implemented or trained drafters and understand what actually drives acceptance rate and end-to-end speed-up.
- A track record of making real inference workloads measurably faster in tokens per second, latency or memory with numbers you can explain.
- Strong command of quantisation and its effect on model quality, including how you measure it.
- Discipline about correctness. You validate numerics against a reference and measure quality, not just speed.
Nice to Have
- Contributions to llama.cpp/ggml, MLX, vLLM, SGLang, ONNX Runtime, OpenVINO or similar.
- Experience with drafter-training frameworks or published work on speculative or parallel decoding.
- Quantisation research or tooling: GPTQ, AWQ, k-quants, MX formats, or quantisation-aware fine-tuning.
- KV cache compression, offloading or paging at the systems level.
- Experience with NPUs (Intel, Qualcomm, AMD, Apple Neural Engine).
What's On Offer
At Opera, you'll join a collaborative, international team with a flat structure, direct communication, and opportunities for professional growth.
- Advanced computing resources: Access to Apple Silicon, NVIDIA (including Blackwell-class), AMD, and Intel hardware, plus cloud GPUs for model training.
- Technical ownership: Lead performance-critical development of Opera's on-device AI, working directly with AI leadership.
- Research opportunities: Contribute to open-source projects and publish technical findings where appropriate.