LLM Production Engineer - Cost, Scale & Observability

Systems Limited; Malaysia; On-site; MYR 120,000 - 240,000; Full time

Job summary

Systems Limited in Malaysia is seeking an experienced MLOps Engineer to own production serving and scaling for LLM/agentic workloads, focusing on inference infrastructure, load balancing, and caching.

You will monitor costs, build observability for failure modes, manage model rollout and incident response, and work with GenAI Engineers on production readiness and cost governance. This role requires strong collaboration with cross-functional teams and a keen eye for efficiency.

Qualifications

• 4–6 yrs platform/MLOps engineering with hands-on LLM/GenAI production experience.
• Deep understanding of LLM inference economics — token costs, batching, caching, model routing.
• Experience with LLM observability tooling (tracing, eval pipelines, prompt/version management).
• Familiarity with multiple model hosting platforms and cost/performance tradeoffs — Azure AI Foundry, AWS Bedrock, Vertex AI, and self-hosted options (vLLM, TGI).
• Experience building canary/rollback strategies for probabilistic systems.
• Comfortable with higher unpredictability of agentic workloads vs classical ML serving.
• Cost-conscious communicator — explain token-cost dynamics to client finance stakeholders.
• Collaborates closely with GenAI Engineers without a hard line between build and run.

Responsibilities

• Own production serving and scaling for LLM/agentic workloads (inference infra, load balancing, caching).
• Monitor and control inference cost — token usage, retry/loop cost, model routing decisions.
• Build observability for LLM-specific failure modes: hallucination rate, latency spikes, prompt drift.
• Manage model/version rollout strategy (canary releases, fallback models, A/B testing).
• Own incident response for LLM/agent production issues.
• Partner with GenAI Engineers and Agentic AI Architects on production-readiness reviews.
• Explain token-cost dynamics to client finance/business stakeholders.
• Collaborate closely with GenAI Engineers without needing a hard line between build and run.
• Support the practice in setting cost governance policy for LLM workloads.

Skills

MLOps engineering

LLM inference economics

Observability tooling

Model hosting platforms

Canary/rollback strategies

Cost-conscious communication

GenAI collaboration

Open-source options (vLLM, TGI)

Tools

Azure AI Foundry

AWS Bedrock

Google Vertex AI

vLLM

TGI

Job description

Systems Limited in Malaysia is seeking an experienced MLOps Engineer to own production serving and scaling for LLM/agentic workloads, focusing on inference infrastructure, load balancing, and caching.

You will monitor costs, build observability for failure modes, manage model rollout and incident response, and work with GenAI Engineers on production readiness and cost governance. This role requires strong collaboration with cross-functional teams and a keen eye for efficiency.