Principal Engineer, On-Device Inference & GPU Kernels

Opera

Województwo mazowieckie

On-site

PLN 200,000 - 350,000

Full time

7 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Advanced computing resources
Technical ownership
Research opportunities

Job summary

Opera is seeking a high-impact engineer to advance on-device AI. You will optimize speculative decoding, write and fuse kernels for attention, and push performance across Metal/MLX, CUDA, Vulkan and other backends.

Join an international team to shape memory hierarchies, quantisation schemes and regression gates, and contribute to open-source-style research while delivering real-world speed-ups on consumer hardware.

Qualifications

  • Deep GPU programming experience in CUDA, Metal/MLX or Vulkan compute.
  • Hands-on speculative decoding experience: implemented or trained drafters and know what drives rate and speed.
  • A track record of speeding up inference workloads in tokens/sec, latency or memory with measurable results.
  • Strong command of quantisation and its impact on model quality and measurement.

Responsibilities

  • Speculative decoding, training and shipping draft models.
  • Write, fuse and tune kernels for attention, quantised matmul, MoE routing and draft verification across backends.
  • Design mixed-precision schemes for weights/activations with hardware formats and cost/quality trade-offs.
  • Push time-to-first-token and tokens-per-second with efficient attention, prefill and kernel scheduling.
  • Build hierarchical KV/context caching across GPU memory, RAM and SSD with eviction and prefetching.
  • Create test matrices, telemetry and regression gates to catch slow drivers or numerical issues.
  • Expose hooks into runtime (activations/attention) for research and security while preserving speed.

Skills

GPU programming
CUDA
Metal/MLX
Vulkan compute
Speculative decoding
Performance tuning

Tools

llama.cpp/ggml
MLX
ONNX Runtime
OpenVINO
vLLM

Job description

Role & Responsibilities
  • Speculative decoding. Train, tune and ship draft models for the models we run block-diffusion drafters (DFlash2-like), semi-autoregressive drafters with confidence-scheduled verification (DSpark), EAGLE-style heads and multi-token prediction and keep pushing acceptance length up and drafting cost down on consumer hardware.
  • Kernels. Write, fuse and tune kernels for attention, quantised matmul, MoE routing, sampling and draft verification, across Metal/MLX, CUDA, Vulkan and OpenVINO.
  • Quantisation. Design mixed-precision schemes for weights, activations and KV cache. Match formats to the hardware, from 4-bit integer on integrated GPUs and NPUs to FP8, FP4 and microscaling formats on newer GPUs, and prove what each one costs in quality before it ships.
  • Prefill and decode. Push time-to-first-token and tokens per second with flash and paged attention, efficient handling of sliding-window and hybrid attention, chunked prefill and kernel-level scheduling.
  • Tiered memory. Build hierarchical KV and context caching that spans GPU memory, system RAM and SSD, with prefix reuse, smart eviction and prefetching that hides I/O, so long contexts and multiple models fit on machines that shouldn't be able to hold them.
  • Making it work everywhere. Build the test matrix, performance telemetry and regression gates that catch a slow driver, a wrong numerical result or a stalled UI before users do.
  • Instrumentation. Expose hooks into the runtime, such as activations and attention, so research and security work can see inside models without giving up the speed of fused kernels.
Job Requirements
  • Deep, hands-on GPU programming in at least one of CUDA, Metal/MLX or Vulkan compute, plus evidence that you've carried that work over to a second backend.
  • Hands-on speculative decoding experience: you've implemented or trained drafters and understand what actually drives acceptance rate and end-to-end speed-up.
  • A track record of making real inference workloads measurably faster in tokens per second, latency or memory with numbers you can explain.
  • Strong command of quantisation and its effect on model quality, including how you measure it.
  • Discipline about correctness. You validate numerics against a reference and measure quality, not just speed.
Nice to Have
  • Contributions to llama.cpp/ggml, MLX, vLLM, SGLang, ONNX Runtime, OpenVINO or similar.
  • Experience with drafter-training frameworks or published work on speculative or parallel decoding.
  • Quantisation research or tooling: GPTQ, AWQ, k-quants, MX formats, or quantisation-aware fine-tuning.
  • KV cache compression, offloading or paging at the systems level.
  • Experience with NPUs (Intel, Qualcomm, AMD, Apple Neural Engine).
What's On Offer

At Opera, you'll join a collaborative, international team with a flat structure, direct communication, and opportunities for professional growth.

  • Advanced computing resources: Access to Apple Silicon, NVIDIA (including Blackwell-class), AMD, and Intel hardware, plus cloud GPUs for model training.
  • Technical ownership: Lead performance-critical development of Opera's on-device AI, working directly with AI leadership.
  • Research opportunities: Contribute to open-source projects and publish technical findings where appropriate.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Engineer, On-Device Inference & GPU Kernels
Principal Engineer, On-Device Inference & GPU Kernels

Opera • Wrocław

Hybrid
PLN 220,000 - 420,000
Principal On-Device AI Inference & GPU Kernel Architect
Principal On-Device AI Inference & GPU Kernel Architect

Opera • Województwo mazowieckie

On-site
PLN 200,000 - 350,000
Advanced computing resources
Technical ownership
Research opportunities
Principal Engineer: On-Device AI Inference & GPU Kernels
Principal Engineer: On-Device AI Inference & GPU Kernels

Opera • Wrocław

Hybrid
PLN 220,000 - 420,000
Principal Engineer, AI Security & Model Internals
Principal Engineer, AI Security & Model Internals

Opera Göteborg • Wrocław

Hybrid
PLN 220,000 - 260,000
Senior ML Engineer | Kimchi (Inference Stack Optimization)
Senior ML Engineer | Kimchi (Inference Stack Optimization)

Cast AI • Polska

Remote
PLN 466,000 - 700,000
Equity options
Remote-friendly
Learning budget
+3
CUDA Engineer
CUDA Engineer

MWDN Ltd. • Województwo mazowieckie

Remote
PLN 180,000 - 240,000
29 days PTO
10 recovery days
Free English classes
+3
AIOps Engineer
AIOps Engineer

Diverse Consulting Group • Polska

Hybrid
PLN 260,000 - 380,000
Private medical care
Co-financing for sports card
Training & learning opportunities
+1
AIOps Engineer
AIOps Engineer

DCG (Poland) • Polska

On-site
PLN 240,000 - 360,000
Private medical care
Co-financing for sports card
Training & learning opportunities
+1
CUDA / AI Engineer
CUDA / AI Engineer

MWDN Ltd. • Województwo mazowieckie

Remote
PLN 120,000 - 180,000
PTO 29 days
Free English classes
HR support
+1
AIOps Engineer
AIOps Engineer

DCG (Poland) • Województwo mazowieckie

On-site
PLN 180,000 - 240,000
Private medical care
Sports card subsidy
Training opportunities
+1
Top locations