AI Platform Engineer

e-Hireo; Bengaluru; On-site; INR 400,000 - 650,000; Full time

Job summary

e-Hireo is hiring a Track A Software / Platform Engineer in Bengaluru to own end-to-end design, build, test, ship, monitor, and iterate on production-grade LLM/ML infrastructure. You will work on model-serving, gateways, and orchestration for scalable workloads, emphasizing reliability and safety in a regulated fintech context.

You will design data and RAG pipelines, build guardrails and evaluation harnesses, and partner with cross-functional teams to produce measurable outcomes.

Qualifications

• Strong CS fundamentals (data structures, algorithms, system design).
• Clear communication with a bias toward reliability and correctness.
• Nice to have fintech / regulated domain experience; LLMOps / MLOps tooling experience.

Responsibilities

• Build and operate model-serving, gateway, and orchestration infra for LLM/ML workloads.
• Design data and RAG pipelines with fresh, consistent ingestion and embeddings.
• Create guardrails, evaluation harnesses, prompt/version management, and observability.
• Improve retrieval and RAG quality: chunking, embeddings, re-ranking.
• Develop agent/workflows and evaluate models with offline and online evals.
• Fine-tune or distill models to reach reliability; productionize with platform.
• Demonstrate production-quality Python and end-to-end service ownership.
• Collaborate across product, data, risk, and platform to deliver measurable outcomes.

Skills

LLM/ML systems

Go

Python

Java

Distributed systems

Observability

Reliability mindset

Tools

LLM frameworks

Vector databases

Kubernetes

Job description

• Own work end-to-end — design, build, test, ship, monitor, iterate — with production ownership and on-call.
• Write clean, well-tested, maintainable code and clear design docs; apply SOLID and sound API design.
• Treat reliability, latency, cost, and observability as first-class requirements, not afterthoughts.
• Build for a regulated environment: data privacy, access controls, auditability, safe handling of customer data.
• Collaborate across product, data, risk, and platform to turn ambiguous problems into measurable outcomes.

Track A — Software / Platform Engineer

• Build and operate model-serving, gateway, and orchestration infra (routing, caching, rate-limiting, fallbacks) for LLM/ML workloads.
• Design data and RAG pipelines — ingestion, chunking, embedding jobs, vector/index stores — that stay fresh and consistent.
• Build guardrails, evaluation harnesses, prompt/version management, and observability (tracing, metrics, cost attribution); harden for scale and failure.
• We look for: production backend/distributed systems in a strong language (Go, Java, Python…); solid concurrency, APIs, databases, queues, and cloud infra; a reliability mindset. Deep ML theory not required.
• Improve retrieval and RAG quality — chunking, embeddings, re-ranking, grounding — measured against real metrics.
• Build agent and prompt workflows; systematically evaluate models, prompts, and pipelines with offline and online evals.
• Fine-tune, adapt, or distill models where it clearly beats prompting; partner with the platform track to productionize to the same reliability bar.
• We look for: production-quality Python and service ownership (not just notebooks); hands-on LLMs, embeddings/retrieval, and evaluation, plus one of fine-tuning, RAG, or agent frameworks; rigor with data and experiments.

Common bar & nice-to-haves

• Strong CS fundamentals (data structures, algorithms, system design) and a track record of shipping in production.
• Clear communication and a bias for reliability and correctness — especially important in fintech.
• Nice to have: fintech / lending / payments or other regulated domains; LLMOps / MLOps tooling, vector DBs, eval frameworks; open-source or AI/ML side projects.