Senior AI Systems Engineer: Scalable Inference | Remote

Boodlebox

United States

Remote

USD 175,000 - 185,000

Full time

8 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Unlimited PTO
Medical, dental, and vision coverage
401(k)
Fully remote work environment

Job summary

BoodleBox is seeking a Senior Software Engineer for AI Systems & Scalability. You will own production backends in Python and Scala, optimize latency and throughput, and explore self-hosted inference to reduce costs.

You’ll operate in a fully remote environment across the United States, contributing to a GenAI platform used by 100,000+ users. You will focus on designing scalable services, monitoring, and cost-aware architectures while collaborating with teams to deploy robust AI capabilities.

Qualifications

  • Proficient in production backend coding in Python and Scala.
  • Experience diagnosing distributed systems under load.
  • Experience serving a model behind an API with concurrency controls.

Responsibilities

  • Production services in Scala and Python with robust monitoring.
  • Improve scalability and reduce latency under load.
  • Evaluate and implement self-hosted inference architectures.
  • Maintain practical MLOps for production models without training pipelines.
  • Own deployed AI services with clear ownership and backups.
  • Track load, cost per request, and latency metrics before/after optimizations.

Skills

Python
Scala
MLOps
Distributed systems
API exposure

Tools

Kubernetes

Job description

BoodleBox is seeking a Senior Software Engineer for AI Systems & Scalability. You will own production backends in Python and Scala, optimize latency and throughput, and explore self-hosted inference to reduce costs.

You’ll operate in a fully remote environment across the United States, contributing to a GenAI platform used by 100,000+ users. You will focus on designing scalable services, monitoring, and cost-aware architectures while collaborating with teams to deploy robust AI capabilities.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Infra & Systems Engineer — Remote
Senior AI Infra & Systems Engineer — Remote

Jobot • Atlanta (GA)

Remote
USD 170,000 - 200,000
Fully Remote
Equity
Healthcare benefits
+1
Senior Software Engineer — AI Systems & Scalability
Senior Software Engineer — AI Systems & Scalability

Boodlebox • United States

Remote
USD 175,000 - 185,000
Unlimited PTO
Medical, dental, and vision coverage
401(k)
+1
Senior AI Context & Retrieval Systems Engineer (Remote)
Senior AI Context & Retrieval Systems Engineer (Remote)

Boodlebox • United States

Remote
USD 175,000 - 185,000
Competitive salary
Unlimited PTO
Health coverage
+2
Senior AI Backend Engineer — Scale & Reliability (Remote)
Senior AI Backend Engineer — Scale & Reliability (Remote)

ByteHire Limited. • New York (NY), Boca Raton (FL)

Remote
USD 171,000 - 209,000
Bonus structure
Generous PTO
Health, dental, vision
+3
Senior Backend Engineer - AI Infrastructure, Remote
Senior Backend Engineer - AI Infrastructure, Remote

Essence Ventures • San Francisco (CA)

Remote
USD 200,000 - 250,000
In-person hubs SF/NYC
Flexible PTO with US holidays
100% health insurance for policyholder
+6
Senior Systems Engineer, AI Inference Platform
Senior Systems Engineer, AI Inference Platform

OpenAI, Inc. • San Francisco (CA)

On-site
USD 180,000 - 260,000
Senior Distributed Systems Engineer — AI Inference Platform
Senior Distributed Systems Engineer — AI Inference Platform

Baseten • San Francisco (CA)

On-site
USD 180,000 - 240,000
Equity
Comprehensive medical for US
Flexible PTO
+4
Remote AI Backend Engineer: Scalable, Production-Ready
Remote AI Backend Engineer: Scalable, Production-Ready

Blue Signal LLC. • Savannah (GA)

Remote
USD 140,000 - 190,000
Fully remote nationwide
Competitive compensation
Staff Software Engineer, Scalable Inference Infrastructure
Staff Software Engineer, Scalable Inference Infrastructure

Anthropic Limited • Seattle (WA)

Hybrid
USD 320,000 - 485,000
Competitive compensation and benefits
Equity donation matching (optional)
Generous vacation and parental leave
+2
Senior AI Systems Engineer — Remote, Scalable Platforms
Senior AI Systems Engineer — Remote, Scalable Platforms

ContentBuffer • San Francisco (CA)

Hybrid
USD 208,000 - 315,000