Senior Infrastructure Security Engineer

fal - Features & Labels; San Francisco (CA); On-site; USD 180,000 - 250,000; Full time

Benefits offered by this job

Health, dental, and vision insurance (

Visa sponsorship and relocation to San

Regular team events and offsites

Interesting and challenging work

Significant learning and growth

Job summary

Cybersecurity Jobs is seeking an experienced Security Engineer to secure fal.ai’s compute platform, spanning Linux kernel, virtualization, and GPU driver layers. You will build robust isolation for shared GPU infrastructure, drive security improvements, and reduce real-world risk across the stack.

You will implement threat models, validate GPU isolation controls, and coordinate with Detection & Response, Infra, and ML teams to ensure secure defaults and rapid mitigations.

Qualifications

• 5+ years in systems or security engineering focusing on the Linux kernel, hypervisors, or device drivers.
• Fluent in kernel C; proficient with Rust, Go, or Python for tooling.
• Strong virtualization internals knowledge (KVM, VFIO, IOMMU/DMA).
• Experience shipping production hardening at fleet scale.

Responsibilities

• Develop and maintain a threat model for executing code on shared GPU hosts, covering containers and microVMs down to PCIe, IOMMU/DMA.
• Design and validate GPU isolation controls, including memory scrubbing and device-state scrubbing between tenants.
• Support deployment of hardware-backed isolation, confidential computing, and attestation.
• Harden kernel builds and host images using config minimization, LSMs, seccomp, and namespace/cgroup policy.
• Reduce attack surface across GPU components including drivers, CUDA runtimes, and NVLink/NVSwitch fabrics.
• Assess and triage security vulnerabilities in kernel, hypervisors, GPU drivers, and firmware; drive fixes or mitigations.
• Monitor vendor bulletins; patch kernel and drivers without fleet disruption.
• Perform vulnerability research on key dependencies.
• Contribute to host and isolation security roadmap and tooling.
• Coordinate with Detection & Response on telemetry; serve as escalation point for host, kernel, and driver incidents.
• Review security-relevant designs from Infra, Platform, and ML teams to make secure approaches default.
• Collaborate with GPU vendors, cloud providers, and upstream maintainers on disclosures and fixes.

Skills

Kernel C

Rust

Go

Python

KVM

VFIO

Tools

KVM

VFIO

IOMMU

Job description

In this role, you will secure fal.ai’s compute platform beneath the application layer, spanning Linux kernel and virtualization components through GPU drivers, firmware, and host isolation. The focus is on building robust isolation for shared GPU infrastructure, reducing real-world risk, and driving security improvements across the stack.

Key Responsibilities

• Develop and maintain a threat model for executing code on shared GPU hosts, covering containers and microVMs down to PCIe, IOMMU, and GPU passthrough.
• Design and validate GPU isolation controls, including memory scrubbing and device-state scrubbing between tenants.
• Support deployment of hardware-backed isolation where appropriate, including confidential computing and attestation.
• Harden kernel builds and host images using config minimization, LSMs, seccomp, and namespace and cgroup policy, with secure and measured boot.
• Reduce attack surface across GPU-related components, including GPU drivers, CUDA runtimes, NVLink/NVSwitch fabrics, BMC, and firmware.
• Assess and triage security vulnerabilities in the kernel, hypervisors, GPU drivers, and firmware, including judging exploitability and real impact, setting severity, and driving fixes or mitigations.
• Monitor vendor bulletins and upstream disclosures, and run kernel and driver patching that keeps pace without disrupting the fleet.
• Perform vulnerability research on key dependencies.
• Contribute to the host and isolation security roadmap and evaluate relevant tooling.
• Coordinate with Detection & Response on telemetry and serve as the escalation point for host, kernel, and driver incidents.
• Review security-relevant designs from Infra, Platform, and ML teams, helping make the secure approach the default.
• Collaborate with GPU vendors, cloud providers, and upstream maintainers on disclosure processes and fixes.

Required Qualifications

• 5+ years in systems or security engineering with deep, hands-on experience in the Linux kernel, hypervisors, or device drivers.
• Fluency in kernel C, with comfort using Rust, Go, or Python for tooling.
• Strong understanding of virtualization internals, including KVM, VFIO, and IOMMU/DMA protection, plus tradeoffs across containers, sandboxes, and microVMs.
• Experience shipping hardening measures that run in production at fleet scale.
• Proven ability to find, triage, or fix real vulnerabilities in low-level code and explain the exploit path.
• Ability to communicate risk clearly to both engineering teams and executives.

Technologies You Will Work With

Linux kernel, hypervisors, microVM, PCIe, IOMMU, GPU passthrough, LSMs, seccomp, namespace and cgroup policy, secure and measured boot, GPU drivers, CUDA runtimes, NVLink, NVSwitch, BMC, kernel C, Rust, Go, Python, KVM, VFIO, IOMMU/DMA protection, containers, sandboxes, LLMs, agents, AI coding tools, NVIDIA GPU stack, open kernel modules, GSP firmware, MIG, vGPU, GPU confidential computing, AMD SEV-SNP, Intel TDX, attestation, confidential computing.

Location and Employment

• Location: San Francisco, CA (onsite)
• Employment type: Full time
• Department: Engineering

Compensation

• Salary range: USD 180,000 - 250,000 per year
• Equity: Offers EquityAdditional compensation statement: $180,000 - $250,000 + equity + comprehensive benefits package

Nice-to-Haves / Qualifications

• AI fluency: daily use of LLMs, agents, and AI coding tools for code review, tooling, and vulnerability research.
• NVIDIA GPU stack experience: open kernel modules, GSP firmware, MIG, vGPU, or GPU confidential computing.
• Confidential computing experience: AMD SEV-SNP, Intel TDX, and attestation.
• Experience with published CVEs, talks, or write-ups on kernel or hypervisor security.
• Prior work at a GPU cloud, hyperscaler, or chip vendor.

Benefits

• Health, dental, and vision insurance (US)
• Visa sponsorship and relocation support to San Francisco
• Regular team events and offsites
• Interesting and challenging work
• Significant learning and growth opportunities