Senior Rust Systems Engineer - Edge AI Inference Platform

Webhosting; Austin (TX); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Cloudflare is seeking a high-agency, systems engineer to design and build AI inference infrastructure across its global network. You will tackle sub-second model cold starts, multi-accelerator workload scheduling, and efficient cache management alongside AI/ML engineers and product teams.

You will work on a platform that runs on GPUs and accelerators in many cities, focusing on reliability, performance, and scalability.

Qualifications

• Proven experience in systems engineering and distributed, high-performance systems.
• Expert proficiency in Rust, particularly in an asynchronous environment.
• Hands-on experience with networking and application protocols (e.g., TCP, HTTP, WebSocket).

Responsibilities

• Platform Architecture: Develop and maintain core components of the serverless inference platform for high availability and scalability.
• Optimization & Performance: Improve model scheduling and request routing to reduce latency.
• System Reliability: Drive measurable reliability improvements and mitigate systemic risks.
• Observability: Expand metrics, logging, tracing, and alerts for production issues.
• Technical Leadership: Lead cross-functional technical projects from concept to deployment.
• Mentorship: Guide junior engineers and foster a collaborative engineering culture.

Skills

Rust

Distributed systems

Networking protocols

Asynchronous programming

Tools

Kubernetes

Nomad

Job description

Cloudflare is seeking a high-agency, systems engineer to design and build AI inference infrastructure across its global network. You will tackle sub-second model cold starts, multi-accelerator workload scheduling, and efficient cache management alongside AI/ML engineers and product teams.

You will work on a platform that runs on GPUs and accelerators in many cities, focusing on reliability, performance, and scalability.