Senior Linux Systems Administrator

RapidFort, Inc.; Oregon; Hybrid; USD 125,000 - 165,000; Full time

Job summary

RapidFort, Inc. is seeking a senior Linux systems administrator to own and run its infrastructure—bare‑metal Linux servers in data centers, the virtual machines and containers that run on them, the networks that connect them, and self‑hosted development services such as GitLab and CI, plus AWS resources.

You will keep these systems reliable, secure and documented, plan and carry out changes, and lead the response when something breaks.

Qualifications

• Experience administering Linux systems at scale across data centers and cloud.
• Strong knowledge of networking, virtualization, and security best practices.
• Proficiency with GitLab, Ansible, and DevSecOps tooling.

Responsibilities

• Build, configure, administer and harden Linux systems and services.
• Manage remote fleet of servers, VMs, containers, storage, and backups.
• Plan OS lifecycle, patches, and upgrades; maintain configuration consistency.
• Design, operate and troubleshoot networks, DNS, and cloud resources (AWS).
• Lead incident response, document runbooks, and improve monitoring.

Skills

Linux systems

Networking

Storage

Virtualization

Monitoring

AWS

GitLab

Security

Scripting Bash

Python automation

Tools

GitLab

Ansible

AWS Console

Docker

KVM

Job description

Location: Remote — United States

Department: Information Technology — Infrastructure Engineering

Reports to: Infrastructure Engineering Manager

Employment Type: Full-time, individual contributor

Experience: 8+ years of relevant hands‑on experience

Base Salary: $125,000–$165,000 annually, plus equity and benefits

Eligibility: U.S. citizenship required

ABOUT RAPIDFORT

RapidFort helps enterprises and government organizations reduce softwaresupply-chain risk through hardened container images, runtime-basedattack surface reduction, and automated vulnerability remediation. Weown and operate the infrastructure that supports our engineering teamsand core systems.

ABOUT THE ROLE

RapidFort is looking for a senior Linux systems administrator to own andrun its infrastructure: bare-metal Linux servers in data centers, thevirtual machines and containers that run on them, the networks thatconnect them to each other and to our staff, self-hosted developmentservices such as GitLab and CI, and our resources in AWS.You will keep these systems reliable, secure and documented, plan andcarry out changes to them, and lead the response when something breaks.This is a hands‑on senior individual‑contributor role with directresponsibility for technical execution. We need someone whose knowledgeruns from disks and kernels up to routing tables, firewalls and DNS.

RESPONSIBILITIES

• Linux systems administration- Build, configure, administer and harden Linux systems: installation,system services, packages, users, permissions, authentication andaccess control- Operate a fleet of remotely managed physical Linux servers and thevirtual machines and containers on them: storage, capacity planningand hardware replacement- Own the operating-system lifecycle: patch testing, staged deployment,operating-system and kernel upgrades, tracking exceptions, and movingsystems off releases that reach end of life- Keep configurations consistent, and find and correct configurationdrift
• Networks, cloud and development services- Design, run and troubleshoot the network: routing, traffic shaping,firewalls, DHCP and DNS, and the overlay network that connects thedata centers, the cloud and remote staff.- Manage our resources in AWS, mainly EC2 and S3.- Run the self-hosted development services, including GitLab, CIrunners, build machines and package mirrors
• Troubleshooting and incident ownership- Independently diagnose production problems involving CPU, memory,disk I/O, filesystems, systemd, networking and service dependencies,such as failed services, boot problems, resource exhaustion, packageconflicts and intermittent performance issues.- Use logs, metrics and command-line tools to find root causes and toconfirm that a fix worked.- Monitor the systems, and improve monitoring and alerting so thatproblems are caught early without needless noise.- Lead incident response, contribute to incident reviews, and completethe corrective actions that keep a problem from recurring.- Work with developers, IT, data‑center staff and hardware vendors whena problem crosses system boundaries.- Escalate when needed, with clear findings, the supporting evidence,and a record of the troubleshooting already done
• Automation and configuration management- Write, debug and maintain Bash scripts for provisioning, diagnostics,remediation, reporting and routine operations, and Python automationwhere it makes operational work more reliable or simpler.- Write and maintain Ansible roles and playbooks to the team'sstandards.- Replace repetitive manual tasks with reliable, maintainableautomation.- Keep configuration and automation in git, test changes, and makechanges through merge requests, review and automated deploymentrather than by editing live systems
• Security and operational reliability- Own access control and security: SSH keys and hardware security keys,network access policies, least privilege, security baselines, timelypatching, vulnerability scanning, audits of exposed services, andsoftware supply‑chain risk.- Run the backups and prove that they restore.- Plan changes and maintenance with a written procedure that gives thesteps, how to check the result, and a way back.- Write and maintain documentation, including runbooks, configurationrecords and troubleshooting notes, and review other engineers'changes.- Take part in a shared on‑call rotation and in scheduled maintenancewindows

REQUIRED EXPERIENCE AND SKILLS

• Linux systems- Linux processes, services, users and permissions, packagemanagement, filesystems and logging, in depth.- Strong experience with Debian and other Linux distributions.- Boot and kernel: UEFI, GRUB, initramfs, kernel configuration andbuilding, kernel modules (including DKMS) and sysctl.- Resource control and diagnosis: cgroups v2, memory management and theout‑of‑memory killer, pressure‑stall information (PSI), and toolssuchas strace, perf, iostat, ss and /proc.- Both systemd and sysvinit
• Networking- TCP/IP fundamentals: IPv4 and IPv6, subnetting, bridges, VLANs,bonding and NAT, and troubleshooting connectivity.- Linux routing: multiple routing tables, policy routing with ip rule,source‑based routing, and routing between sites.- Traffic shaping with tc: queueing disciplines (HTB, fq_codel, cake),classification with tc filters or firewall marks, and ingress shapingthrough ifb.- nftables, including sets, maps and connection tracking, and enoughiptables to read and migrate older rule sets.- DNS and DHCP: unbound as a validating recursive resolver with localzones and forwarding, dnsmasq for DHCP and local DNS, split‑horizonDNS, and debugging name resolution with dig.- WireGuard, and NetBird or a similar WireGuard‑based overlay networksuch as Tailscale or Headscale: peers, groups, access policies,routing peers for subnets, and DNS. Experience self‑hosting thecontrol plane is a plus.- Reading packet captures with tcpdump or Wireshark
• Storage- ZFS in depth: pool layout, datasets and zvols, snapshots, send andreceive, scrubs and resilvering, ARC tuning, and recovering a degradedor damaged pool.- mdadm RAID, LVM, ext4 and XFS, filesystem repair, SSD caching such asbcache or dm-cache, and handling failing disks (SMART, NVMe health)
• Virtualization and containers- KVM/QEMU or Xen, LXC and Docker, and enough Kubernetes to debug acluster.
• Monitoring and backups- Monitoring and alerting, and backup and recovery, including testingthat restores work.
• AWS- EC2: instances, images, security groups, EBS, instance profiles andthe instance metadata service (IMDSv2).- S3: bucket policies, versioning, lifecycle rules and encryption.- IAM roles and policies written to the principle of least privilege.
• Git, GitLab, GitOps and configuration management- Expert git: branching, merging and conflict resolution in amerge-request workflow, without rewriting or destroying history.- GitLab administration: a self‑hosted installation and its upgrades,groups, projects and permissions, protected branches, approval rules,and CI/CD pipelines and runners.- GitOps and infrastructure as code: configuration that lives in git andis applied automatically after review, drift detection, and OpenTofuor Terraform.- Writing and maintaining Ansible roles and playbooks, or substantialexperience with a comparable configuration‑management tool.
• Security- Host hardening, least privilege, firewall design, and finding andclosing services that are exposed by mistake.- SSH in depth: sshd configuration, SSH certificates, and FIDO2 hardwarekeys such as YubiKeys. GPG signing and verification.- TLS certificates and their renewal, and safe handling of secrets.- Patch management, vulnerability scanning, software supply‑chain risk,and investigating incidents from logs.
• Scripting- Expert Bash: robust scripts with sound error handling, quoting, traps,and commands run over SSH, which you can write, troubleshoot andmaintain in production on your own.- Good working Python for systems automation and operational tooling.
• Judgment and communication- A record of personally resolving complex technical problems andfollowing through on the corrective actions.- Sound judgment about change risk, testing, rollback and escalation.- Clear, precise written English for change plans, incident reports,reviews, tickets and documentation.- The ability to work independently within a remote team.

PREFERRED

• Server hardware: IPMI, iDRAC, iLO and other BMC consoles, RAID andHBA controllers, and firmware updates.
• Diagnosing PCIe devices and their drivers and firmware from kernellogs.
• Physical data‑center work, and working with data‑center remote hands.
• GPU servers, NVIDIA drivers and CUDA environments, and infrastructurefor AI workloads.
• Other virtualization platforms, such as Proxmox or VMware.
• SAN and NAS storage, and multipathing.
• Monitoring and host intrusion detection, such as Prometheus, Grafana,netdata or Wazuh.
• Jenkins or other build farms, and caching mirrors or proxies for apt,PyPI, npm or Maven.
• Backup tools such as restic, borg or ZFS replication.
• Infrastructure that supports SOC 2 or ISO 27001 controls.
• Debian and its derivatives.
• Other cloud providers, such as Linode.