LLMops Engineer

Coursera Inc.; India; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Coursera Inc. is seeking an AI Platform and LLMOps Engineer to join Pune team. You will own the operational backbone to run agentic AI systems in production, from prompt/versioning to evaluation and cost management, collaborating with engineers, product managers, and AI specialists.

You will design and maintain tooling for LLM-powered workflows, ensure reliability, and contribute to scalable AI adoption across Coursera’s enterprise customers.

Qualifications

• 4+ years in a software engineering role, backend focus.
• Production ops for LLM-powered systems via APIs/ pipelines.
• Experience with Python/Java/TypeScript/Go; Docker & Kubernetes.

Responsibilities

• Build and maintain tooling for LLM systems, including prompts, versioning, evals, and CI/CD.
• Implement monitoring for production LLMs: latency, cost, quality, drift.
• Run automated evaluations to catch regressions and hallucinations before customers.
• Manage RAG pipelines and vector stores for fresh, accurate data.
• Implement safety guardrails: content filtering, PII redaction, access controls.
• Own cost governance: caching, routing, usage reporting to control spend.
• Collaborate with PMs and senior engineers to scope requirements and build reliable systems.
• Participate in sprint planning and grooming.

Skills

Backend Engineering

DevOps fundamentals

Backend languages

Docker & Kubernetes

APIs & databases

SQL & NoSQL

Automated tests

Cloud platforms

Terraform

CI/CD tooling

Production debugging

Engineering quality

Education

CS/CE degree

Tools

LangFuse

LangSmith

Pinecone/Weaviate/pgvector

LangGraph

LiteLLM/Open Router

Job description

Coursera and Udemy are now one company, bringing together two mission-driven brands to create the world’s most powerful platform for turning learning into progress. Together, we help more than 300 million learners and 12,000+ enterprise customers build the skills they need for a world being reshaped by AI. Read more about the combined company by visiting our blog .

AI is transforming how people learn, work, and grow, and the need for new skills has never been greater. Coursera brings trusted content and credentials from leading university and industry partners, while Udemy brings a dynamic skills marketplace and global network of real-world experts. By combining these strengths, we can connect more people and organizations to the skills they need, when they need them.

Shape what comes next

By joining our team, you’ll have the opportunity to reshape how the world learns and applies skills—and help millions of people participate in the new economy. Bring your ideas, expertise, and perspective to meaningful work that can make a difference at global scale.

About the Team (Job Location - Pune) :

As an AI Platform and LLMOps Engineer, you will join a fast-paced innovation team at Coursera working on AI enablement for our organization and customers. As an internal focus area, this team builds and maintains the infrastructure & tooling required to drive meaningful AI adoption across the organization, providing a platform for non-technical team members to build and deploy tools and transforming entire business functions into AI-native operating models. As a customer-facing focus area, this team builds custom AI-powered solutions tailored for our enterprise customers supporting them in their journey of AI adoption and upskilling beyond just the content on our platforms.

You will own the operational backbone that lets our agentic AI systems run reliably in production — from prompt and pipeline versioning to evaluation, monitoring, incident and cost management. You will work closely with a cross-functional team of Software Engineers, AI Specialists and Product Managers – helping turn promising AI prototypes into reliable, observable, and cost-effective production systems.

Key Responsibilities

• Build and maintain the operational tooling for our LLM-powered systems, including prompt/pipeline versioning, evaluation harnesses, and CI/CD workflows tailored to non-deterministic AI outputs
• Implement monitoring and observability for production LLM systems — tracking latency, token usage, cost per request, output quality, and drift over time
• Design and run automated evaluation suites to catch regressions, hallucinations, and quality degradation before they reach customers
• Manage RAG pipelines and vector store infrastructure, keeping retrieval sources fresh, accurate, and performant
• Implement safety and compliance guardrails — content filtering, PII redaction, and access controls — in line with enterprise data privacy and residency requirements
• Own cost governance for LLM usage: caching strategies, model routing, and usage reporting to keep spend predictable as adoption scales
• Collaborate closely with Product Managers and Senior Engineers to scope operational requirements and translate them into reliable systems
• Participate in sprint planning, technical grooming, and retrospective discussions

Education

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering or a related field

Basic Qualifications

• 4+ years of experience in a software engineering role, with a solid background in Backend Engineering and DevOps fundamentals
• Hands-on experience operating or supporting LLM-powered systems in production (via APIs, RAG pipelines, frontier/open weight/fine-tuned models)
• Proficiency in a backend language such as Python, Java, TypeScript or Go, and familiarity with Docker and container orchestration (Kubernetes)
• Experience implementing APIs, working with SQL and NoSQL databases, and writing automated tests
• Working knowledge of at least one cloud platform (AWS, GCP, or Azure), across both managed and self-hosted services
• Familiarity with infrastructure-as-code (e.g., Terraform) and CI/CD tooling (e.g., GitHub Actions, Jenkins)
• Comfort debugging production issues involving non-deterministic systems, with attention to detail and a bias towards reliability
• Strong belief in engineering quality and building tooling that creates leverage for others

Preferred Qualifications

• Experience with LLMOps-specific tooling such as LangFuse, LangSmith, Weights & Biases-style evaluation frameworks, or vector databases like Pinecone, Weaviate, pgvector
• Working knowledge of RAG architectures, MCP implementation & governance, agent orchestration frameworks like LangGraph and LLM Gateways such as LiteLLM, Open Router & enterprise AI ecosystems such as Vertex AI, Bedrock
• Exposure to prompt management and versioning practices treated as code along with model access management, deterministic guardrails and evals
• Understanding of AI governance considerations — data privacy, residency, and compliance in enterprise AI deployments
• Ability to work in a fast-paced, ambiguous environment with a proactive, ownership-driven mindset
• Strong communication skills and comfort collaborating across engineering, product, and strategy functions

Why Join Us?

• Work at the operational core of Coursera's AI transformation — solving problems that keep real production AI systems reliable, safe, and cost-effective
• Build hands-on expertise in one of the fastest-growing and most in-demand engineering disciplines
• Join a supportive, innovative team with a strong culture of continuous learning and improvement
• Be part of a mission-driven company transforming global access to education and upskilling in the AI era

Coursera is an Equal Opportunity Employer committed to building a welcoming and inclusive workplace. We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request at [e-mail].

COURSERA IS A U.S. GOVERNMENT CONTRACTOR AND WE ARE REQUIRED TO INQUIRE ABOUT CONFLICTS BEFORE WE CAN RECRUIT CURRENT OR FORMER GOVERNMENT EMPLOYEES. IF THIS APPLIES TO YOU, WE REQUEST A LETTER FROM YOUR SUPERVISOR THAT THEY HAVE BEEN INFORMED ABOUT YOUR CONTACT WITH COURSERA (OR AS A FORMER EMPLOYEE, A LETTER STATING AN ETHICS OPINION WHICH IDENTIFIES ANY LIMITATIONS WHICH APPLY TO YOUR WORK IN THE PRIVATE SECTOR). THE LETTER SHOULD IDENTIFY ANY CONFLICTS OR POTENTIAL CONFLICTS, IF ANY EXIST.