Computer Vision Engineer

SGS Consulting; Redmond; On-site; AUD 215,000 - 272,000; Full time

Job summary

Reality Labs Research is seeking a ML Software Engineer to bring state‑of‑the‑art machine learning from the lab to embedded hardware. You will train and adapt models, optimize and port them to constrained devices, and profile on-device performance, collaborating with researchers to achieve real-time AR/VR outcomes.

The role requires hands-on experience with PyTorch or TensorFlow, strong C/C++ skills, and a track record of optimizing ML workloads for embedded targets.

Qualifications

• Industry experience optimizing and running ML inference on embedded or resource-constrained devices.
• Experience training ML models using deep learning frameworks such as PyTorch or TensorFlow.
• 2+ years of industry experience writing C/C++.
• Experience profiling and debugging performance issues across the ML model and the target hardware.
• Experience working in cross-functional teams.

Responsibilities

• Work side by side with research scientists to take ML models from research prototypes to efficient, real-time inference on embedded devices.
• Train, fine-tune, and adapt ML models with deployment constraints (latency, memory, power, compute) in mind.
• Optimize models for on-device inference using techniques such as quantization, pruning, distillation, operator fusion, and architecture adjustments.
• Port, integrate, and profile ML models on embedded targets (e.g. DSPs, NPUs, microcontrollers, mobile SoCs), identifying and resolving performance bottlenecks.
• Write clean, maintainable, high-performance C/C++ code for on-device inference pipelines and integrate it into real-time prototype systems.
• Build tools and infrastructure that make it faster for researchers to evaluate model performance and accuracy on target hardware.
• Coordinate with internal hardware, runtime, and toolchain teams across RL to evaluate and integrate with existing systems.
• Work collaboratively as part of a multifunctional team where communication, documentation, and teamwork are highly valued.
• Learn constantly, dive into new areas with unfamiliar technologies, and embrace the ambiguity of AR/VR problem solving.

Skills

On-device ML optimization

PyTorch/TensorFlow

C/C++

Embedded systems

Profiling & debugging

Tools

ExecuTorch

TFLite/LiteRT

ONNX Runtime

Job description

The Reality Labs Research (RLR) Team brings together a world‑class team of researchers, developers, and engineers to create the future of virtual and augmented reality, which together will become as universal and essential as smartphones and personal computers are today. We are developing all the technologies needed to enable breakthrough AR glasses, including optics and displays, computer vision, audio, graphics, brain‑computer interfaces, haptic interaction, eye/hand/face/body tracking, perception science, and true telepresence.

The RLR Audio team believes that by redefining human hearing and developing auditory machine intelligence, our research will overcome the barriers of distance, noise, and hearing ability to enhance human connection and communication. Through our Superhuman Communication and Connection initiative, we develop and mature audio technologies for AR glasses and wearables, from early research through tech transfer to the product teams that ship them.

We are seeking a ML Software Engineer to bring state‑of‑the‑art machine learning out of the lab and onto the device. Research models only matter to our prototypes if they run in real time, within tight compute, memory, and power budgets, on the embedded hardware that powers wearables. This role sits at that boundary: you will train and adapt models alongside research scientists, then own the work of optimizing, porting, and running them efficiently on embedded targets.

The ideal candidate is equally comfortable in a PyTorch training loop and in a C/C++ inference runtime on a resource‑constrained device. They understand that on‑device ML is a co‑design problem, where model architecture, quantization, and the target hardware all shape one another, and they can make the trade‑offs required to get a research model running on real hardware quickly. Familiarity with audio ML, computer vision ML, or audio DSP is a strong plus, as many of our models operate on multimodal (audio and visual) sensor data.

Our team cultivates an honest and considerate environment where self‑motivated individuals thrive. We encourage a strong sense of ownership and embrace the ambiguity that comes with working on the frontiers of research.

We have plenty of researchers who know how to train models, but limited expertise in optimizing and running them on embedded devices. The immediate need is optimizing speech extraction models for on‑device inference, but RLR Audio has an ongoing need for this expertise beyond that first project.

A Typical Day involves working closely with research scientists to understand a new model and its deployment constraints, then owning the path to getting it running efficiently on target hardware: training or adapting the model, optimizing, and porting it, profiling on‑device, and iterating. You share knowledge and expertise with teammates, contribute directly to technical solutions, and actively participate in team tech meetings.

Minimum Qualifications

• Industry experience optimizing and running ML inference on embedded or resource‑constrained devices
• Experience training ML models using deep learning frameworks such as PyTorch or TensorFlow
• 2+ years of industry experience writing C/C++
• Experience profiling and debugging performance issues across the ML model and the target hardware
• Experience working in cross‑functional teams

Top 3 must-have HARD skills:

• Optimizing and running ML inference on embedded / resource‑constrained devices
• Training ML models (PyTorch or similar)
• C/C++ (2+ years industry experience)

Preferred Qualifications

• Experience with ML for audio (e.g., speech enhancement, source separation, keyword spotting, acoustic event detection)
• Experience with ML for computer vision (e.g., detection, tracking, segmentation, pose estimation)
• Experience with audio DSP fundamentals (e.g., filtering, STFT/filterbanks, beamforming, real‑time audio processing)
• Experience with on‑device inference frameworks and toolchains (e.g. ExecuTorch, TFLite/LiteRT, ONNX Runtime, vendor NPU/DSP SDKs)
• Experience with model compression techniques such as quantization‑aware training, post‑training quantization, pruning, or distillation
• Experience writing hardware‑specific optimizations (e.g., SIMD/NEON, DSP intrinsics, fixed‑point arithmetic)
• Experience working in the AR/VR or wearable device space
• Ability to gather requirements and work closely with researchers to develop novel solutions
• History of writing code to support the execution of research initiatives

Good to have skills:

Responsibilities

• Work side by side with research scientists to take ML models from research prototypes to efficient, real‑time inference on embedded devices
• Train, fine‑tune, and adapt ML models with deployment constraints (latency, memory, power, compute) in mind
• Optimize models for on‑device inference using techniques such as quantization, pruning, distillation, operator fusion, and architecture adjustments
• Port, integrate, and profile ML models on embedded targets (e.g. DSPs, NPUs, microcontrollers, mobile SoCs), identifying and resolving performance bottlenecks
• Write clean, maintainable, high‑performance C/C++ code for on‑device inference pipelines and integrate it into real‑time prototype systems
• Build tools and infrastructure that make it faster for researchers to evaluate model performance and accuracy on target hardware
• Coordinate with internal hardware, runtime, and toolchain teams across RL to evaluate and integrate with existing systems
• Work collaboratively as part of a multifunctional team where communication, documentation, and teamwork are highly valued
• Learn constantly, dive into new areas with unfamiliar technologies, and embrace the ambiguity of AR/VR problem solving

How will performance be measured: While work is collaborative in nature, we’re looking for motivated individuals who can take a goal("get this model running in real time on this device"), form a plan, make progress independently, raise concerns proactively, and provide input into overall team goals.