Senior Distributed Systems Engineer for Inference Platform

Hedra Inc.; San Francisco (CA); On-site; USD 150,000 - 260,000; Full time

Benefits offered by this job

Competitive compensation and equity

401k

Healthcare (Silver PPO Medical, Vision

Lunch and snacks at the office

Job summary

Hedra Inc. in San Francisco is seeking a Senior or Staff Software Engineer with deep expertise in building and operating distributed production systems.

You will work on Hedra’s inference infrastructure, designing and operating systems that schedule, route, and manage compute-intensive workloads across diverse resources. The role emphasizes reliability, performance, and cost efficiency, requiring you to reason from first principles and drive thoughtful tradeoffs.

Qualifications

• 5+ years of software engineering experience, with significant experience building distributed backend or infrastructure systems.
• A track record of designing and operating high-availability production services at meaningful scale.
• Strong distributed systems fundamentals, including concurrency, queues, retries, failure recovery, consistency, backpressure, capacity, and load behavior.
• Experience debugging complex production systems across multiple stack layers.
• Strong judgment around architectural tradeoffs, particularly reliability, performance, complexity, and operational cost.
• Experience with CI/CD, comprehensive testing and validation strategies, monitoring, alerting, and production operations.
• Strong programming fundamentals and experience in systems-oriented backend languages; Python experience is helpful.
• Experience using agentic coding tools and workflows beyond basic prompting.
• Comfort operating in underspecified environments with independent decision-making.
• Ability to communicate technical decisions clearly and collaborate across teams.

Responsibilities

• Design, build, and operate distributed systems that power Hedra’s inference infrastructure.
• Build systems for scheduling, routing, and managing compute-intensive workloads across heterogeneous resources.
• Improve throughput, latency, reliability, and resource utilization across our serving infrastructure.
• Design systems that remain predictable and resilient under load, partial failures, changing capacity, and unpredictable workloads.
• Own production systems end to end, including deployment, CI/CD, testing and validation, observability, monitoring, alerting, debugging, and incident response.
• Identify architectural bottlenecks and failure modes and drive solutions.
• Work across infrastructure, model serving, APIs, and developer-facing systems as the platform evolves.
• Use modern agentic coding workflows as part of how you design, build, debug, and ship software.
• Help shape the technical direction of a small engineering organization where individual engineers have substantial ownership.

Skills

Distributed systems

Backend engineering

CI/CD

Python

Performance optimization

Observability

Cloud infrastructure

System design

Tools

Kubernetes

Job description

Hedra Inc. in San Francisco is seeking a Senior or Staff Software Engineer with deep expertise in building and operating distributed production systems.

You will work on Hedra’s inference infrastructure, designing and operating systems that schedule, route, and manage compute-intensive workloads across diverse resources. The role emphasizes reliability, performance, and cost efficiency, requiring you to reason from first principles and drive thoughtful tradeoffs.