Gen AI Inference Engineer (LLM, Kubernetes, Production)

Quantum Technologies LLC

Austin (TX)

On-site

USD 86,000 - 123,000

Full time

6 days ago
Be an early applicant

Get a reply from this recruiter

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Quantum Technologies LLC in Austin, TX is seeking a Software Engineer (Gen AI Solutions) to design, build, and optimize production LLM inference services powering GenAI features. The role focuses on deploying scalable inference services on Kubernetes, reducing latency and cost, and integrating with external model APIs.

2+ years in ML/LLM in production, strong Python, and experience with LLM serving frameworks are required.

Qualifications

  • 2+ years of software engineering in ML/LLM systems in production.
  • Strong Python skills plus Go, C++, or Rust for performance-critical paths.
  • Hands-on experience with LLM serving frameworks and model deployment.

Responsibilities

  • Design, deploy, and maintain scalable inference services for LLMs on Kubernetes.
  • Profile and reduce latency and cost per request.
  • Benchmark quality vs latency vs throughput.
  • Integrate with hosted LLM APIs and internal orchestration.
  • Add observability across request paths with logs, metrics, traces, dashboards.
  • Set up evaluation/regression pipelines to protect output quality.
  • Plan capacity, autoscale, monitor GPU utilization and cost.
  • Write design docs and handover knowledge.
  • Deliver production-ready inference services with CI/CD and IaC.
  • Produce latency and cost benchmark reports.
  • Create dashboards and alerts.
  • Create architecture docs and runbooks.

Skills

Python
ML/LLM systems
Go/C++/Rust
REST gRPC

Tools

vLLM
TensorRT-LLM
TGI
SGLang
Triton Inference Server
Kubernetes
Docker

Job description

Quantum Technologies LLC in Austin, TX is seeking a Software Engineer (Gen AI Solutions) to design, build, and optimize production LLM inference services powering GenAI features. The role focuses on deploying scalable inference services on Kubernetes, reducing latency and cost, and integrating with external model APIs.

2+ years in ML/LLM in production, strong Python, and experience with LLM serving frameworks are required.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Gen AI Inference Engineer - Scalable LLM Serving
Gen AI Inference Engineer - Scalable LLM Serving

CloudBigD • Austin (TX)

On-site
USD 86,000 - 123,000
Gen AI Solutions Engineer - Production LLM Inference
Gen AI Solutions Engineer - Production LLM Inference

Robotics Technologies • Jackson (TN)

On-site
USD 86,000 - 123,000
Gen AI Inference Engineer (Production-Ready)
Gen AI Inference Engineer (Production-Ready)

Valiant Technologies LLC • Austin (TX)

On-site
USD 96,000 - 124,000
Gen AI Inference Engineer — Scalable LLM Services
Gen AI Inference Engineer — Scalable LLM Services

Ethereum Technologies LLC • Austin (TX)

On-site
USD 86,000 - 123,000
Senior AI Solutions Engineer (LLM & RAG)
Senior AI Solutions Engineer (LLM & RAG)

Quantum Technologies. LLC • St. Louis (MO)

On-site
USD 120,000 - 190,000
Generative AI Engineer: LLM, NLP & Deployment
Generative AI Engineer: LLM, NLP & Deployment

Quantum Technologies LLC • Raleigh (NC)

On-site
USD 120,000 - 180,000
Gen AI Inference Engineer for Scalable LLMs
Gen AI Inference Engineer for Scalable LLMs

Robotic Process Automation LLC • Austin (TX)

On-site
USD 86,000 - 123,000
Applied AI Engineer: LLMs, Retrieval & MLOps
Applied AI Engineer: LLMs, Retrieval & MLOps

Quantum Technologies. LLC • Austin (TX)

Hybrid
USD 120,000 - 160,000
Equal opportunity employer
AI/ML Engineer - NLP, Generative AI & LLMs
AI/ML Engineer - NLP, Generative AI & LLMs

Quantum Technologies LLC • City of Albany (NY)

On-site
USD 120,000 - 180,000
GenAI Lead Engineer: LLMs, Production Systems
GenAI Lead Engineer: LLMs, Production Systems

Quantum Technologies. LLC • Atlanta (GA)

On-site
USD 180,000 - 240,000