Gen AI Solutions Engineer - Production LLM Inference

Robotics Technologies

Jackson (TN)

On-site

USD 86,000 - 123,000

Full time

2 days ago
Be an early applicant

Get a reply from this recruiter

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Robotics Technologies LLC in Austin, TX is seeking an experienced Software Engineer (Gen AI Solutions) to design, build, and optimize production LLM inference services powering GenAI features for internal and customer-facing applications. Collaborate with ML, platform, and product teams to make model serving faster, cheaper, and more reliable at scale.

You will implement CI/CD, infrastructure as code, and observability, while profiling latency, throughput, and cost and documenting architectures

Qualifications

  • Software engineer with 2+ years in production ML/LLM systems.
  • Strong Python and working knowledge of performance languages.
  • Hands-on with LLM serving frameworks and GPU-accelerated deployments.
  • Experience with distributed systems, API design, and observability.

Responsibilities

  • Design, deploy, and maintain scalable inference services for open-weight and hosted LLMs on Kubernetes-based infrastructure.
  • Profile and reduce end-to-end latency and cost per request.
  • Apply model and serving optimizations, then benchmark trade-offs between quality, latency, and throughput.
  • Build and maintain integrations with external model APIs and internal orchestration frameworks, including routing, fallback, retries, and rate-limit handling.
  • Add observability across the request path, covering logging, metrics, tracing, and dashboards for request volume, downstream call ratios, errors, and token usage.
  • Set up evaluation and regression pipelines so model or serving changes don't degrade output quality.
  • Do capacity planning and autoscaling, and track GPU utilization and cost.
  • Write technical design docs and runbooks, and hand knowledge over to the internal team.
  • Production-ready inference service(s) with CI/CD and infrastructure as code
  • Latency and cost benchmark reports, measured before and after optimization
  • Monitoring dashboards and alerting
  • Architecture docs, runbooks, and handover sessions

Skills

Python
Go/C++/Rust
LLM serving frameworks
Distributed systems
API design

Tools

Kubernetes
Docker
GPU scheduling
PyTorch
Hugging Face

Job description

Robotics Technologies LLC in Austin, TX is seeking an experienced Software Engineer (Gen AI Solutions) to design, build, and optimize production LLM inference services powering GenAI features for internal and customer-facing applications. Collaborate with ML, platform, and product teams to make model serving faster, cheaper, and more reliable at scale.

You will implement CI/CD, infrastructure as code, and observability, while profiling latency, throughput, and cost and documenting architectures

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Gen AI Inference Engineer - Scalable LLM Serving
Gen AI Inference Engineer - Scalable LLM Serving

CloudBigD • Austin (TX)

On-site
USD 86,000 - 123,000
Gen AI Inference Engineer (Production-Ready)
Gen AI Inference Engineer (Production-Ready)

Valiant Technologies LLC • Austin (TX)

On-site
USD 96,000 - 124,000
Gen AI Inference Engineer — Scalable LLM Services
Gen AI Inference Engineer — Scalable LLM Services

Ethereum Technologies LLC • Austin (TX)

On-site
USD 86,000 - 123,000
Gen AI Inference Engineer (LLM, Kubernetes, Production)
Gen AI Inference Engineer (LLM, Kubernetes, Production)

Quantum Technologies LLC • Austin (TX)

On-site
USD 86,000 - 123,000
Gen AI Inference Engineer for Scalable LLMs
Gen AI Inference Engineer for Scalable LLMs

Robotic Process Automation LLC • Austin (TX)

On-site
USD 86,000 - 123,000
Production ML Engineer — LLMs, Agentic AI & Inference
Production ML Engineer — LLMs, Agentic AI & Inference

INSPYR Solutions • Sunnyvale (CA)

Hybrid
USD 120,000 - 180,000
Senior Generative AI Engineer — LLM Production & Backend
Senior Generative AI Engineer — LLM Production & Backend

Robotics Technologies • Plano (TX)

On-site
USD 150,000 - 210,000
AI/ML Engineer: Generative AI & LLM Solutions
AI/ML Engineer: Generative AI & LLM Solutions

Robotic Process Automation LLC • Plano (TX)

On-site
USD 120,000 - 160,000
Senior AI Engineer — Flexible Production LLMs & Gen AI
Senior AI Engineer — Flexible Production LLMs & Gen AI

Blend360 • United States

Hybrid
USD 140,000 - 210,000
AWS Certifications
Databricks Training
Snowflake Training
+4
Senior AI/ML Engineer for Production-Grade LLM AI Apps
Senior AI/ML Engineer for Production-Grade LLM AI Apps

MassGenics, Inc. • Jersey City (NJ)

On-site
USD 83,000 - 90,000
Medical & pharmacy coverage
Dental/vision insurance
401(k)
+5