EMBEDDED OS ENGINEER

Tom Trese Networking LLC; Salt Lake City (UT); On-site; USD 120,000 - 180,000; Full time

Job summary

Tom Trese Networking LLC is seeking a hands-on Embedded OS Engineer to own the Ubuntu-based Linux firmware stack on our MK2 drone platform (Qualcomm QRB5165). You will build, harden, and ship the OS through automated CI/CD pipelines, tackling low-level OS internals and Yocto BitBake workflows.

Responsibilities include maintaining the Yocto build system, writing .bb recipes, optimizing builds with QEMU and native ARM64, performing OS hardening, configuring systemd services and network stacks, and

Qualifications

• Experience building Ubuntu-based Linux firmware for embedded devices.
• Strong knowledge of Yocto/BitBake and related tooling.
• Proficient in headless, flash-and-go hardware OS development.
• Experience with CI/CD pipelines and automated testing.
• Security hardening and patch management for embedded systems.

Responsibilities

• Own and maintain the MK2 build workflow and BSP integration.
• Write and maintain .bb/.bbappend recipes and patches.
• Debug BitBake tasks and optimize build performance.
• Configure systemd services, network setup, and device boot flow.
• Implement OS hardening, post-processing, and security reporting.

Skills

Embedded OS

Yocto/BitBake

CI/CD pipelines

Jenkins

Linux firmware

Systemd

Networking basics

Security hardening

Troubleshooting

Tools

Yocto

BitBake

QEMU

Docker

systemd

dpkg/apt

Nessus

Grafana

Prometheus

Job description

Our client is looking for a hands-on Embedded OS Engineer to own the Ubuntu-based Linux firmware stack on our MK2 drone platform (Qualcomm QRB5165). You will be the person who builds the OS, keeps it secure, makes it reliable, and ships it through automated CI/CD pipelines.

This is not a role for someone who has only worked at the application layer. We need someone comfortable navigating low-level OS internals, Yocto’s BitBake machinery, system unit files, and the realities of headless embedded hardware with no display and limited recovery options.

Yocto / BitBake Build System

Own and maintain the teal-mk2-build repository, including layer configuration, local.conf tuning, and image recipes that produce the qti-ubuntu-robotics-image.

Write and maintain .bb recipes and .bbappend overlays; create .patch files (git diff-based) against the pinned Qualcomm QRB5165 BSP (LU.UM.3.3.1) so that upstream sources are never modified directly.

• Debug BitBake task failures - understanding the do_fetch do_compile do_rootfs do_image pipeline, sstate-cache behavior, and when to use -c cleansstate vs. -c rootfs -f.
• Optimize build performance: reduce full-build times by understanding QEMU emulation overhead vs. native ARM64 compilation, tuning BB_NUMBER_THREADS and PARALLEL_MAKE, and leveraging shared sstatecache across Jenkins agents.
• Manage package inclusion, exclusion (IMAGE_INSTALL:remove, PACKAGE_EXCLUDE, BAD_RECOMMENDATIONS), and inter-recipe dependency graphs.

Rootfs Post-Processing & OS Hardening

Implement ROOTFS_POSTPROCESS_COMMAND functions to perform postbuild OS customization: package upgrades, apt security patching (Ubuntu ESM), service installation, and file system fixups.

• Remove unnecessary services from production images - including HTTP servers (lighttpd), TFTP daemons, and other attack-surface-expanding services identified in Blue List / Nessus security assessments.
• Write bash scripts and systemd service units that execute reliably in a headless, rootfs post-install context (no interactive terminal, limited /proc and /dev availability).
• Manage the dpkg / apt ecosystem within the embedded rootfs: pinning packages, handling held packages, validating package state, and ensuring apt lock files are clean across incremental builds

Jenkins CI/CD Pipelines

Maintain and improve Jenkins pipelines that build Yocto firmware images, including handling concurrent build isolation, artifact staging, and race conditions between parallel jobs.

Diagnose and resolve Jenkins agent performance issues: Java heap tuning, Docker overlay2 disk pressure, container lifecycle management, and build environment reproducibility.

• Implement proper artifact copy patterns (stageDir isolation) to prevent race conditions when multiple builds run concurrently against shared directories.
• Manage GitLab repository structure and branch protection for the build repo; implement .patch-based change workflows that preserve the integrity of pinned upstream BSP branches

Embedded Linux System Configuration

• Configure systemd-networkd for runtime network mode switching (DHCP linklocal) on headless devices using udev rules, button-press event handlers, and LED feedback mechanisms.
• Write and maintain systemd service units and timers: understanding WantedBy, After, Requires, and ExecStart semantics for embedded boot sequences.
• Implement reliable USB logging and file transfer services (usb-gadget, udevadm) for field diagnostics on devices with no screen.
• Develop and maintain Prometheus node_exporter integrations and processexporter configurations for drone fleet health monitoring via Grafana dashboards.

Security & Compliance

Reliability & Performance Tuning

• Remediate Nessus / Blue List security findings affecting the embedded OS: SSH hardening (key-only auth, sshd_config tuning), open port reduction, and service inventory documentation.
• Maintain /etc/shadow hygiene, locked account policies, and PAM configurations appropriate for production embedded devices.
• Generate and maintain security compliance artifacts (port inventories, service lists, patch status reports) for internal security assessors.