Software Engineer, Infrastructure

Fudo Partners

New York (NY)

On-site

USD 180,000 - 240,000

Full time

12 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical, dental and vision coverage
Flexible paid time off
Winter break
Parental leave
Equity participation
Retirement savings plan

Job summary

Fudo Partners is seeking an Infrastructure Software Engineer to design, build, and scale core systems behind a high-performance ML inference platform. You’ll work across infrastructure, distributed systems, and ML operations to run production AI workloads efficiently across cloud and GPU environments.

The role emphasizes hands-on engineering, strong software fundamentals, and the challenge of scaling AI infrastructure. Join a technically ambitious team driving AI at scale.

Qualifications

  • Experience with Python or Go in production systems.
  • Knowledge of cloud providers (AWS, GCP, Azure) and cloud-native architecture.
  • Experience with Kubernetes, Docker and container orchestration.
  • Familiarity with distributed systems and fault tolerance.
  • Exposure to ML infrastructure or production AI systems.

Responsibilities

  • Design, build and maintain infrastructure software for production ML workloads.
  • Develop backend services using Python and Go.
  • Build and improve Kubernetes-based deployment and orchestration systems.
  • Scale ML models deployment and serving infrastructure.
  • Develop systems for resource allocation and workload scheduling.
  • Improve performance, scalability, reliability of distributed infrastructure.
  • Build monitoring and observability tooling for production AI systems.
  • Collaborate with cross-team engineering groups.

Skills

Python
Go
Distributed systems
Backend engineering
Cloud infrastructure

Tools

Kubernetes
Docker
AWS
GCP
Azure

Job description

Infrastructure Software Engineer | AI Infrastructure | ML Systems | Inference

We're representing an ambitious, venture-backed AI technology company building the infrastructure that enables next-generation AI applications to operate at scale.

The company operates at the intersection of machine learning infrastructure, distributed systems, cloud computing and high-performance model inference. Its engineering platform enables AI teams to deploy, serve and scale increasingly sophisticated models in production environments.

This is an opportunity to work on the underlying systems powering modern AI, solving complex infrastructure challenges across compute, orchestration, performance and reliability.

The Role

We're looking for an Infrastructure Software Engineer to help design, develop and scale the core systems behind a high-performance ML inference platform.

You'll work across infrastructure engineering, distributed systems and machine learning operations, building software that enables production AI workloads to run efficiently across cloud and GPU environments.

This is a hands-on engineering position suited to someone with strong software engineering fundamentals, an interest in complex infrastructure problems and a desire to work closer to the systems powering AI.

The team is working across several technically demanding areas, including:

  • Distributed infrastructure and multi-cloud resource management
  • GPU infrastructure and accelerated computing
  • High-performance LLM inference and model serving
  • Multi-node distributed inference architectures
  • Kubernetes-based orchestration and container scheduling
  • Compute allocation, capacity planning and resource optimisation
  • Infrastructure automation and deployment systems
  • Observability, monitoring and production reliability
  • Scaling machine learning workloads across cloud environments
Responsibilities
  • Design, build and maintain infrastructure software supporting production machine learning workloads.
  • Develop backend infrastructure services using Python and Go.
  • Build and improve Kubernetes-based deployment and orchestration systems.
  • Contribute to the infrastructure responsible for deploying, managing and scaling machine learning models.
  • Develop systems for resource allocation, workload scheduling and compute optimisation.
  • Improve the performance, scalability and reliability of distributed infrastructure.
  • Build monitoring, observability and operational tooling for production AI systems.
  • Support infrastructure automation and improve deployment workflows.
  • Collaborate with engineers across infrastructure, platform and applied machine learning teams.
  • Identify opportunities to improve system architecture, engineering efficiency and operational performance.
Technical Experience

We're interested in engineers with experience or exposure to several of the following:

  • Software Engineering: Python, Go, backend engineering and systems programming.
  • Cloud Infrastructure: AWS, GCP, Azure and cloud-native architecture.
  • Containerisation: Kubernetes, Docker and container orchestration.
  • Distributed Systems: Distributed computing, service architecture, concurrency and fault tolerance.
  • ML Infrastructure: Model deployment, model serving, ML platforms and production machine learning systems.
  • AI Inference: LLM inference, inference optimisation, model execution and serving infrastructure.
  • Compute Infrastructure: GPU workloads, resource management and accelerated computing.
  • Observability: Monitoring, logging, metrics and production infrastructure reliability.
What We're Looking For
  • Strong software engineering fundamentals and the ability to build production-quality systems.
  • Experience with at least one modern programming language, ideally Python or Go.
  • Working knowledge of Kubernetes, containers and cloud infrastructure.
  • An understanding of distributed systems and infrastructure architecture.
  • Exposure to machine learning infrastructure, model deployment or production AI systems.
  • An interest in GPU computing, inference infrastructure and the engineering challenges associated with scaling AI.
  • A practical, problem-solving mindset and an appetite for tackling technically complex challenges.
  • Strong communication skills and the ability to work effectively within a collaborative engineering environment.
  • Competitive salary and meaningful equity participation.
  • Comprehensive medical, dental and vision coverage for employees and eligible dependants (US).
  • Flexible paid time off and company-wide winter break.
  • Paid parental leave.
  • Family-building and fertility benefits.
  • Company-supported retirement savings plan (US).
  • Exposure to a technically ambitious engineering environment working at the forefront of AI infrastructure and machine learning systems.
Interested?

If you're an infrastructure engineer interested in distributed systems, GPU computing, ML infrastructure and the next generation of AI inference platforms, we'd be interested in speaking with you.

Apply directly or get in touch for a confidential conversation.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Founding Engineer, AI Infrastructure
Founding Engineer, AI Infrastructure

Piris Labs • San Francisco (CA)

On-site
USD 100,000 - 200,000
Equity
401(k)
Health insurance
+1
Inference Engineer - Member of Technical Staff
Inference Engineer - Member of Technical Staff

Designworks Talent • Bellevue (WA)

Hybrid
USD 150,000 - 230,000
Medical, dental, and vision insurance
401(k) with company match
Paid holidays
Infrastructure Operations Engineer
Infrastructure Operations Engineer

lightningai • United States

Remote
USD 160,000 - 200,000
Medical, dental, vision
401(k) matching
Unlimited PTO
+4
Software Engineer (Backend-Focused) $140,000 - $225,000 Posted 12 hours ago
Software Engineer (Backend-Focused) $140,000 - $225,000 Posted 12 hours ago

Fuel Talent • Seattle (WA)

Remote
USD 140,000 - 225,000
AI Training Infrastructure - Member of Technical Staff
AI Training Infrastructure - Member of Technical Staff

Designworks Talent • Bellevue (WA)

Hybrid
USD 150,000 - 210,000
Medical, dental, and vision insurance
401(k) plan with company match
Paid holidays
AI Infrastructure Specialist
AI Infrastructure Specialist

Ali Aydi • United States

Remote
USD 95,000 - 155,000
100% remote work
Health insurance
Paid time off
Senior Software Engineer, Machine Learning Infrastructure & Automation
Senior Software Engineer, Machine Learning Infrastructure & Automation

fal-ai • United States

Remote
USD 150,000 - 210,000
Health insurance
Dental insurance
Vision insurance
AI Platform Support Engineer: Kubernetes, GPUs & ML Infra
AI Platform Support Engineer: Kubernetes, GPUs & ML Infra

Lightning AI • New York (NY)

Hybrid
USD 115,000 - 140,000
Health coverage
Equity (RSUs)
401(k) matching
+3
Back End Engineer, AI Systems
Back End Engineer, AI Systems

Salt • United States

On-site
USD 140,000 - 190,000
Member of Technical Staff
Member of Technical Staff

Harrison Clarke • San Francisco (CA)

On-site
USD 180,000 - 280,000