Senior Embedded Software Engineer - Inference AI/ML

Socket.dev; Austin (TX); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Competitive salary

ACS Equity Package

Health, Dental, Vision Insurance

Paid Time Off

Job summary

Allen Control Systems is seeking a Senior Embedded Software Engineer - Inference AI/ML to own end-to-end deployment of trained models on edge hardware. You will build and optimize the C++ infrastructure for on-device inference, handling pre/post-processing, memory management, and real-time constraints.

You will work with CV/ML, firmware, and product teams to hit performance targets, profiling GPUs/NPUs/DSPs/FPGA and refining models for latency, memory, and power.

Qualifications

• 10+ years in embedded software or systems engineering with ML deployment on edge devices.
• Strong C++ expertise and CUDA knowledge, with hands-on ML framework experience.
• Experience optimizing models for latency, memory, and power on constrained hardware.
• Familiarity with edge platforms (e.g., NVIDIA Jetson) and Linux/RTOS.

Responsibilities

• Own end-to-end ML model deployment on resource-constrained edge hardware.
• Design, implement, and optimize C++ infrastructure for on-device inference.
• Profile latency, memory, and power; tune models to meet constraints.
• Build tooling for packaging, versioning, and OTA updates to devices.
• Collaborate with CV/ML, firmware, and product teams to set targets.

Skills

C++ proficiency

CUDA

PyTorch

Edge inference runtimes

Computer vision

Education

Bachelor’s or Master’s in CS/EE

Tools

TensorRT

ONNX Runtime

TensorFlow Lite

CUDA-enabled GPUs

Job description

Company Overview

Allen Control Systems (ACS) is a cutting-edge defense startup founded by two former Navy electrical engineers with a proven track record in robotics and software. We are developing an autonomous gun turret using advanced computer vision and control systems to precisely detect, track, and neutralize enemy drones.

With an engineering-first culture, ACS values technical excellence and innovation. Backed by our founders’ successful exits from two previous ventures acquired for a combined $180M in 2022, we are committed to ensuring that the groundbreaking technologies we develop will have a real-world impact.

About The Role

We are looking for a Senior Embedded Software Engineer - Inference AI/ML to own the end-to-end process of taking trained ML models and deploying them efficiently onto resource-constrained edge hardware. This role sits at the intersection of machine learning, embedded systems, and hardware engineering.

You will integrate, convert, and optimize models to run within strict constraints on latency, memory, power, and thermal budget, and build the supporting C++ infrastructure that hosts them on device. You will partner closely with the CV/ML Engineering team who build the models, the Embedded and Firmware teams who own the device, and the product team who define performance targets. Success means models that are not just accurate in the lab but fast, small, and dependable in the field.

What You’ll Do

• Apply quantization, pruning, knowledge distillation, operator fusion, and graph optimization to shrink models and reduce inference cost while protecting accuracy; convert trained models into edge-deployable formats using ONNX and TensorRT.

• Profile inference on target accelerators including GPUs, NPUs, DSPs, and FPGAs; measure latency, throughput, memory footprint, and power consumption, then drive the changes needed to hit performance targets.

• Design, write, and maintain the C++ application code that hosts inference on device, including pre- and post-processing pipelines, data and memory management, threading, and interfaces to the rest of the embedded system; ensure the combined model and C++ stack meets real-time constraints and fits within device memory budget.

• Build test harnesses to verify on-device accuracy against reference results and catch regressions from optimization or quantization; contribute to tooling for packaging, versioning, and delivering model updates to deployed devices.

• Set best practices for edge deployment, review designs and code, and mentor other engineers on optimization and embedded ML techniques; work closely with research, firmware, and product teams to set realistic performance targets and feed hardware constraints back into model design.

What You’ll Need

• 10+ years of professional embedded software or systems engineering experience, including at least 2 years focused on deploying ML models to embedded or edge devices; Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, Computer Engineering, or equivalent practical experience.

• Very strong C++ proficiency; working knowledge of CUDA; hands-on experience with PyTorch and at least one edge inference runtime such as TensorFlow Lite, ONNX Runtime, or TensorRT.

• Practical experience with model optimization techniques including post-training quantization, quantization-aware training, pruning, and distillation; demonstrated ability to profile and optimize for latency, memory, and power on constrained hardware.

• Working knowledge of embedded or edge platforms such as NVIDIA Jetson, Qualcomm, ARM Cortex, or comparable NPUs and SoCs, and of Linux or an RTOS; solid grasp of computer architecture concepts relevant to inference including memory hierarchy, fixed-point arithmetic, and accelerator offload; domain experience in computer vision or sensor processing on device.

You’ll Stand Out

• Hands-on experience deploying computer vision models for detection or tracking tasks on embedded or edge hardware.

• Experience with NVIDIA Jetson specifically, including TensorRT optimization and deployment on Jetson platforms.

• Background in defense, autonomous systems, or robotics where real-time reliability matters.

• Experience building or contributing to model update and OTA delivery pipelines for deployed edge devices.

What We Offer

• Competitive salary

• ACS Equity Package

• Health, Dental, Vision Insurance

• Paid Time Off

Allen Control Systems is an Equal Opportunity Employer, providing equal employment opportunities to all employees and applicants for employment. Allen Control Systems prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. #LI-AS1