Founding Systems Engineer: Low-Latency LLM & Edge Infra

Berkley Hunt; New York (NY); On-site; USD 150,000 - 210,000; Full time

Job summary

Berkley Hunt is partnering with a well-funded startup building next-gen infrastructure for high-performance, low-latency LLM inference at scale. This founding-level role focuses on low-level systems, distributed environments, and mission-critical performance challenges.

The engineer will own global routing, backend infra, and security controls, shaping edge-based inference and scalable production systems across global locations.

Qualifications

• Experience building distributed, concurrent systems focused on speed and reliability.
• Deep expertise in systems programming (Rust, C++, or Zig).
• Strong Python skills for automation and control-plane development.
• Experience designing infrastructure for LLM inference or high-scale CDN environments.
• Proficiency with AWS, Postgres, Redis, and Kafka.
• Familiarity with Linux internals, performance tuning, and kernel-level debugging.
• Observability tools such as Zipkin or Jaeger.
• Clear, concise communicator with a strong documentation and operational mindset.

Responsibilities

• Architect and implement global routing engines with microsecond decision times.
• Build and manage high-performance backend infrastructure from scratch, focusing on concurrency, throughput, and resilience.
• Own the control plane, monitoring systems, and production documentation for all deployments.
• Lead efforts in internal security and access management systems.
• Design and optimize infrastructure for edge-based inference with maximum hardware utilization and minimal latency.
• Collaborate cross-functionally to drive infrastructure decisions from prototype to production.

Skills

Distributed systems

Systems programming

Python scripting

Cloud services

Observability

Tools

Kafka

PostgreSQL

Redis

AWS

Zipkin/Jaeger

Job description

Berkley Hunt is partnering with a well-funded startup building next-gen infrastructure for high-performance, low-latency LLM inference at scale. This founding-level role focuses on low-level systems, distributed environments, and mission-critical performance challenges.

The engineer will own global routing, backend infra, and security controls, shaping edge-based inference and scalable production systems across global locations.