LLMOps Engineer GenAI Platforms

Tekskills; Bengaluru; On-site; INR 900,000 - 1,400,000; Full time

Job summary

Tekskills Bangalore is seeking an LLMOps Engineer to build GenAI platforms that are secure, scalable, and compliant. You will establish governance, implement security controls, and enable development teams to deploy high-performing GenAI applications across cloud and hybrid environments.

Ideal candidates have 4+ years in MLOps or platform engineering with hands-on LLM experience, across providers like OpenAI, Anthropic, or Azure OpenAI, and a solid grasp of prompt engineering and lifecycle

Qualifications

• Strong MLOps experience
• Hands-on with LLMs for at least 2 years
• Transitioned from MLOps to LLMOps with understanding of LLM lifecycle

Responsibilities

• Design and implement governance frameworks for GenAI platforms, ensuring compliance with internal policies and external regulations (e.g., GDPR, AI Act).
• Define and enforce responsible AI practices including fairness, transparency, explainability, and auditability.
• Implement robust security protocols including IAM, data encryption, secure API access, and model sandboxing.
• Collaborate with security teams to conduct risk assessments and ensure secure deployment of LLMs.
• Build and maintain scalable LLMOps pipelines for model training, fine-tuning, evaluation, deployment, and monitoring.
• Automate model lifecycle management with CI/CD, versioning, rollback, and observability.
• Develop and manage GenAI agents capable of reasoning, planning, and tool use.
• Integrate and orchestrate LLMs from multiple providers to support hybrid and fallback strategies.
• Optimize prompt engineering, context management, and agent memory for production use.
• Ensure high availability, low latency, and cost-efficiency of GenAI workloads across cloud and hybrid environments.
• Implement monitoring and alerting for model drift, hallucinations, and performance degradation.
• Partner with GenAI developers to embed best practices and reusable components (SDKs, templates, APIs).
• Provide technical guidance and documentation to accelerate development and ensure platform consistency.

Skills

MLOps

LLMs

LLMOps

Prompt engineering

Python

Docker

Cloud platforms

Governance

Responsible AI

Security

Vector databases

AI governance frameworks

Education

Bachelor's or Master's in Computer Science/Engineering

Tools

MLflow

LangChain

LlamaIndex

Langraph

OpenAI

Cohere

FAISS

Pinecone

CI/CD tools

Job description

Job Title: LLMOps Engineer GenAI Platforms

Location: Bangalore

Minimum: 3 years of relevant experience

Maximum: 5 years total experience

Must include:

• Strong MLOps experience
• Hands-on with LLMs for at least 2 years OR
• Transitioned from MLOps to LLMOps with understanding of LLM lifecycle

About the Role:

We are looking for a forward-thinking LLMOps Engineer to join our team and help build the next generation of secure, scalable, and responsible Generative AI (GenAI) platforms. This role will focus on establishing governance, security, and operational best practices while enabling development teams to build high-performing GenAI applications. You will also work closely with GenAI agents and integrate LLMs from multiple providers to support diverse use cases.

Key Responsibilities:

• Design and implement governance frameworks for GenAI platforms, ensuring compliance with internal policies and external regulations (e.g., GDPR, AI Act).
• Define and enforce responsible AI practices including fairness, transparency, explainability, and auditability.
• Implement robust security protocols including IAM, data encryption, secure API access, and model sandboxing.
• Collaborate with security teams to conduct risk assessments and ensure secure deployment of LLMs.
• Build and maintain scalable LLMOps pipelines for model training, fine-tuning, evaluation, deployment, and monitoring.
• Automate model lifecycle management with CI/CD, versioning, rollback, and observability.
• Develop and manage GenAI agents capable of reasoning, planning, and tool use.
• Integrate and orchestrate LLMs from multiple providers (e.g., OpenAI, Anthropic, Cohere, Google, Azure OpenAI) to support hybrid and fallback strategies.
• Optimize prompt engineering, context management, and agent memory for production use.
• Ensure high availability, low latency, and cost-efficiency of GenAI workloads across cloud and hybrid environments.
• Implement monitoring and alerting for model drift, hallucinations, and performance degradation.
• Partner with GenAI developers to embed best practices and reusable components (SDKs, templates, APIs).
• Provide technical guidance and documentation to accelerate development and ensure platform consistency.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 4+ years of experience in MLOps, DevOps, or platform engineering, with 1-2 years in LLM/GenAI environments.
• Deep understanding of LLMs, GenAI agents, prompt engineering, and inference optimization.
• Experience with LangChain, LlamaIndex, Langraph or similar agent frameworks.
• Hands-on with MLflow, or equivalent tools.
• Proficient in Python, containerization (Docker) and cloud platforms (AWS/GCP/Azure).
• Familiarity with AI governance frameworks and responsible AI principles.
• Experience with vector databases (e.g., FAISS, Pinecone), RAG pipelines, and model evaluation frameworks.
• Knowledge of Responsible AI, red‑team­ing, and OWASP security priciples.