Senior Software Engineer — AI Systems & Scalability

Boodlebox; United States; Remote; USD 175,000 - 185,000; Full time

Benefits offered by this job

Fully remote work

Competitive base salary

Unlimited PTO

Medical/dental/vision coverage

401(k)

Career growth

Job summary

BoodleBox seeks a Senior Software Engineer to optimize its AI systems and scalability. You’ll own production services in Python and Scala, tackle latency and cost per request, and explore self-hosted inference for embeddings and models.

You'll work in the AI Systems group, deploy end-to-end, and drive architectural choices without being the generalist lead. This fully remote role aligns with our Colorado Springs hub strategies.

Qualifications

• Production backend in Python and Scala.
• Diagnosed a distributed system under load.
• Served a model behind an API with concurrency and reliability.
• Weighing managed vs self-hosted deployment costs and trade-offs.

Responsibilities

• Develop production services in Scala and Python with monitoring and recovery from the start.
• Improve scalability and reduce latency and cost per request.
• Evaluate self-hosted inference architectures and GPU utilization.
• Implement practical MLOps and production-ready services.
• Own AI services in production with clear ownership and backups.
• Contribute across AI and backend where services intersect.

Skills

Python

Scala

Distributed systems

Kubernetes

MLOps

API design

Tools

Kubernetes

Job description

Senior Software Engineer — AI Systems & Scalability

Make BoodleBox's AI platform faster and cheaper per request across Scala and Python, and lead the move to self-hosted inference where the numbers support it.

Remote, United States

About BoodleBox

BoodleBox is a VC-backed GenAI platform built for higher education. With 100,000+ users across 1,200+ colleges and universities, we partner with faculty, staff, and students to make responsible AI adoption practical, impactful, and lasting. Our platform brings together leading AI tools — including ChatGPT, Claude, Gemini, and Perplexity — under one roof, purpose-built for the unique demands of academic environments. We're guided by three core values: service, education, and collaboration.

The Role

You're a backend engineer who is at home in a Scala service and a Python service on the same day. You take it personally when a request is slow. You've stood up a model behind an API under real load, watched the GPU utilization, and can tell us what it cost per request. You have enough MLOps to run models in production and no interest in training them. You find the bottleneck before you ask for more machines.

BoodleBox is already a generative AI platform in production: Scala and Python microservices on Kubernetes, hundreds of thousands of users, models served today through managed providers. We're not building an ML platform or running experiments. We want to make the one we have faster, more scalable, and more flexible about where models run. That includes evaluating and implementing self-hosted inference — embeddings, re-rankers, document-processing models, possibly open-weight LLMs — instead of relying only on providers like Azure AI Foundry, with everything that implies for latency, scaling, and cost.

You'd sit in our AI Systems group alongside a retrieval specialist and the engineer growing into leading that group. You're the specialist on scalability and inference. You'll contribute across the AI and application backend where services meet, through missions with clear owners, without becoming the engineer everyone sends their hard problems to.

This role is fully remote, with a company hub in Colorado Springs, CO.

What You'll Own

• Production services in Scala and Python. The integrations, middleware, and services that connect generative AI to the application platform, built with monitoring and recovery from the start.
• Scalability and latency. Finding and removing bottlenecks in request handling, throughput, and concurrency so the platform handles more users at lower cost per request.
• Self-hosted inference. The evaluation, the serving architecture, GPU utilization, autoscaling, and the honest cost comparison against managed providers — then the implementation if the numbers support it.
• Practical MLOps. Models monitored in production, inference infrastructure versioned and operated reliably. No training pipelines; that's not what we do.
• Your services in production. A set of AI services you steward with a named backup: healthy, documented, safe for other engineers to change.
• Backend work inside missions. Cross-capability projects led by someone else where you own the AI-systems piece.
• The numbers. Load, cost per request, p99 latency — before and after, written down.

Who You Are

• You write production backend code in Python and Scala. Strong in Python, and either strong in Scala or demonstrably quick to get there on the JVM.
• You've diagnosed a distributed system under load. From logs, metrics, and traces, to the actual cause, not to “add replicas.”
• You've served a model behind an API. Concurrency, resource limits, the moment it fell over, what you did.
• You know the managed-versus-self-hosted trade. When it's worth running it yourself and when it isn't, in dollars and in pages.
• You own outcomes. Design through deploy through the 3am follow-through.
• You're comfortable being the specialist, not the lead. Someone else is growing into leading this group. You're here for depth.
• You've built something with an LLM. Not a wrapper. Something with real thought behind the architecture.

We're Not Checking Boxes. We're Looking for the Real Thing.

Didn't come up through a traditional path? Good. Tell us about it.

No hyperscaler logo on your resume? We've hired people who outperformed everyone who did.

Career gap? Weird pivot? Started somewhere completely left field? Bring it. The people who got here the strange way usually have the best instincts.

What we actually want to see: someone who finds the bottleneck before asking for more machines without thinking about it. If that's you, the path you took to get here is just a good story.

Who Thrives Here

We move quickly, and the speed comes from engineers who own their services end to end. You'll deploy your own code. You'll see a latency improvement land in a student's chat the same day. The platform team is small — one DevOps lead, one DevSecOps engineer — which means production stability is real engineering work here, not a ticket you file.

You'll fit if you like both halves of our stack, if cost-per-request is a number you'd check for fun, and if you want your scaling work to matter to a university that just rolled AI out to 30,000 students. We're building for education, in regulated environments. Reliability isn't a feature; it's the product.

What We Offer

• Competitive base salary of $175,000–$185,000, plus bonus and equity.
• Unlimited PTO.
• Medical, dental, and vision coverage. BoodleBox covers 75% of premiums.
• 401(k).
• Fully remote work environment.
• Real career growth. Inside a high-growth, mission-driven company.

About BoodleBox

BoodleBox is a team of curious, collaborative builders committed to creating meaningful experiences for our customers and each other. We don't have legacy processes protecting legacy roles. We have a small team trying to build something genuinely new.

We hire humans first. BoodleBox is built on the belief that AI works best when the people behind it reflect the full range of human experience. This means different backgrounds, perspectives, lived experiences, and yes, unconventional career paths. We're an equal opportunity employer, full stop — no discrimination based on race, religion, national origin, gender identity or expression, sexual orientation, age, marital status, veteran status, or disability status. Need an accommodation during the application or interview process? Reach out — we'll make it work.