Gen AI Inference Engineer — Scalable LLM Services

Ethereum Technologies LLC

Austin (TX)

On-site

USD 86,000 - 123,000

Full time

6 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Ethereum Technologies in Austin, TX is seeking a Software Engineer to design, build, and optimize production LLM inference services powering GenAI features for internal and customer-facing apps. You will collaborate with ML, platform, and product teams to scale model serving, implement observability, and ensure reliable, cost-efficient performance with CI/CD and infrastructure as code.

This role demands hands-on experience in ML systems, Python and performance-oriented languages, and a strong

Qualifications

  • 2+ years of production ML/LLM systems experience.
  • Strong Python and knowledge of Go, C++, or Rust.
  • Experience with LLM serving frameworks and Kubernetes.
  • Familiar with PyTorch and Hugging Face.
  • Experience with REST/gRPC, observability tools, and performance profiling.

Responsibilities

  • Design, deploy, and maintain scalable LLM inference services on Kubernetes.
  • Reduce latency and cost per request while maintaining quality.
  • Integrate with hosted LLM APIs and internal orchestration.
  • Add logging, metrics, tracing, and dashboards for observability.
  • Develop CI/CD and infrastructure as code for production services.
  • Write design docs and runbooks; perform capacity planning and autoscaling.
  • Create evaluation/regression pipelines to ensure quality.

Skills

Python
Go/C++/Rust
PyTorch
Hugging Face
Kubernetes
Docker
GPU scheduling
REST/gRPC
Performance profiling
Observability

Tools

vLLM
TensorRT-LLM
TGI
SGLang
Triton Inference Server
ONNX Runtime
TorchScript
FAISS
Milvus
pgvector
Prometheus
Grafana
OpenTelemetry
Terraform
GitHub Actions

Job description

Ethereum Technologies in Austin, TX is seeking a Software Engineer to design, build, and optimize production LLM inference services powering GenAI features for internal and customer-facing apps. You will collaborate with ML, platform, and product teams to scale model serving, implement observability, and ensure reliable, cost-efficient performance with CI/CD and infrastructure as code.

This role demands hands-on experience in ML systems, Python and performance-oriented languages, and a strong

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Gen AI Inference Engineer - Scalable LLM Serving
Gen AI Inference Engineer - Scalable LLM Serving

CloudBigD • Austin (TX)

On-site
USD 86,000 - 123,000
Gen AI Solutions Engineer - Production LLM Inference
Gen AI Solutions Engineer - Production LLM Inference

Robotics Technologies • Jackson (TN)

On-site
USD 86,000 - 123,000
Gen AI Inference Engineer for Scalable LLMs
Gen AI Inference Engineer for Scalable LLMs

Robotic Process Automation LLC • Austin (TX)

On-site
USD 86,000 - 123,000
Gen AI Inference Engineer (LLM, Kubernetes, Production)
Gen AI Inference Engineer (LLM, Kubernetes, Production)

Quantum Technologies LLC • Austin (TX)

On-site
USD 86,000 - 123,000
Gen AI Inference Engineer (Production-Ready)
Gen AI Inference Engineer (Production-Ready)

Valiant Technologies LLC • Austin (TX)

On-site
USD 96,000 - 124,000
AI Engineer: GenAI & LLM Cloud Deployments
AI Engineer: GenAI & LLM Cloud Deployments

Ethereum Technologies • Dallas (TX)

On-site
USD 120,000 - 180,000
AI/ML Engineer: Scale Generative AI & LLM Apps
AI/ML Engineer: Scale Generative AI & LLM Apps

Ethereum Technologies LLC • Plano (TX)

On-site
USD 120,000 - 180,000
Production ML Engineer — LLMs, Agentic AI & Inference
Production ML Engineer — LLMs, Agentic AI & Inference

INSPYR Solutions • Sunnyvale (CA)

Hybrid
USD 120,000 - 180,000
GenAI ML Engineer: LLMs, RAG & Cloud MLOps
GenAI ML Engineer: LLMs, RAG & Cloud MLOps

Ethereum Technologies • Austin (TX)

On-site
USD 180,000 - 250,000
Lead GenAI ML Engineer — LLMs, Pipelines, Cloud
Lead GenAI ML Engineer — LLMs, Pipelines, Cloud

Ethereum Technologies • Atlanta (GA)

On-site
USD 180,000 - 260,000