Specialist in machine learning engineering

Jobgether

India

Remote

INR 1,500,000 - 2,500,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Fully remote work
Full-time employment
Work on large-scale AI infrastructure
Global collaboration

Job summary

Jobgether partners with a company to recruit a Specialist in Machine Learning Engineering based in India. The role focuses on building and operating scalable AI infrastructure, spanning LLM inference, model serving, vector databases, and RAG platforms.

You will work across SRE, distributed systems, and ML infra in a hybrid multi-cloud environment. You will own reliability and performance of critical AI infrastructure, implement observability dashboards, and drive automation to reduce toil.

Qualifications

  • Bachelor's degree in CS/engineering or related field with 5+ years of experience or Master's with 3+ years' experience.
  • 3+ years of software development in Python, Go, or Java focusing on automation or backend services.
  • Strong experience with Kubernetes, microservices, and CI/CD pipelines.
  • Background in Site Reliability Engineering, production ops, distributed systems troubleshooting, and observability.
  • Experience with Prometheus, Grafana, Splunk, or OpenTelemetry.

Responsibilities

  • Operate, scale, and optimize multi-tenant LLM inference, model-serving, VectorDB/RAG, and MCP platforms across cloud/GPU clusters.
  • Establish and monitor SLOs, SLIs, automated health checks, and observability dashboards.
  • Develop automation for deployment rollouts, autoscaling, canary releases, self-healing systems, and disaster recovery.
  • Optimize GPU utilization and inference performance with Kubernetes, vLLM, and Ray for scheduling and batching.
  • Lead complex production incident responses and conduct root-cause investigations.
  • Collaborate with AI researchers and platform teams for production readiness reviews and reliable service delivery.
  • Participate in on-call rotation and improve operational practices.

Skills

Python
Go
Java
Kubernetes
CI/CD
Observability
Distributed systems
SRE principles
OpenTelemetry

Education

Bachelor's degree in Computer Science/Engineering
Master's degree in a related field

Tools

Prometheus
Grafana
Splunk
OpenTelemetry
vLLM
Ray
KServe
TensorRT-LLM
Milvus
Pinecone
Redis

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Specialist in Machine Learning Engineering based in India.

As a Specialist in Machine Learning Engineering, you will help build, operate, and scale the infrastructure powering advanced AI services and applications. You will work at the intersection of Site Reliability Engineering, distributed systems, and machine learning infrastructure, ensuring that production AI workloads remain reliable, secure, scalable, and cost-efficient. Your responsibilities will span large-scale LLM inference, model serving, vector databases, and retrieval-augmented generation (RAG) platforms. You will also drive automation, improve system observability, and optimize GPU resources to enhance performance and availability. Working closely with AI researchers, software engineers, and platform teams, you will contribute to the development of resilient AI infrastructure in a dynamic, hybrid multi-cloud environment.

Accountabilities

You will take ownership of the reliability, performance, and operational excellence of critical AI infrastructure, helping ensure that large-scale machine learning services operate efficiently in production. Your work will combine hands-on engineering, automation, incident management, and cross-functional collaboration.

  • Operate, scale, and optimize multi-tenant LLM inference, model-serving, VectorDB/RAG, and Model Context Protocol (MCP) platforms across global cloud and GPU clusters.

  • Establish and monitor Service Level Objectives (SLOs), Service Level Indicators (SLIs), automated health checks, and observability dashboards using tools such as Prometheus, Grafana, Splunk, and OpenTelemetry.

  • Develop automation for deployment rollouts, autoscaling, canary releases, self-healing systems, and disaster recovery to reduce operational overhead.

  • Optimize GPU utilization and inference performance by improving throughput, latency, model scheduling, batching, caching, and resource allocation using Kubernetes, vLLM, and Ray.

  • Lead complex production incident responses, conduct root‑cause investigations using methodologies such as 5 Whys, and implement long‑term solutions to prevent recurring issues.

  • Collaborate with AI researchers, product developers, and platform engineering teams to conduct production readiness reviews and ensure reliable service delivery.

  • Participate in a sustainable on‑call rotation and continuously improve operational practices, system resilience, and service availability.

Requirements

The ideal candidate combines strong software engineering expertise with practical experience operating distributed, cloud‑native systems in production environments. You should have a solid understanding of SRE principles, observability, and infrastructure automation, alongside an interest in the challenges of deploying and scaling modern AI workloads.

  • Bachelor's degree in Computer Science, Engineering, or a related field, with at least 5 years of relevant experience, or a Master's degree with at least 3 years of experience.

  • At least 3 years of software development experience using Python, Go, or Java, particularly in building automation tools, infrastructure solutions, or production backend services.

  • Strong experience operating distributed and containerized production systems using Kubernetes, microservices architectures, and CI/CD pipelines.

  • Proven background in Site Reliability Engineering, production operations, distributed systems troubleshooting, and observability.

  • Practical experience with monitoring and observability tools such as Prometheus, Grafana, Splunk, or OpenTelemetry.

  • Working knowledge of at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud Platform (GCP).

  • Hands‑on experience with LLM inference and serving frameworks such as vLLM, TensorRT‑LLM, Ray Serve, or KServe is highly desirable.

  • Familiarity with GPU orchestration, capacity planning, model parallelism, and performance profiling.

  • Experience managing vector databases, search engines, or distributed caching solutions such as Milvus, Pinecone, or Redis.

  • Understanding of generative AI architectures, including RAG pipelines, model routing, Model Context Protocol (MCP), and agent runtimes.

  • Experience designing highly available multi‑region systems, automated remediation solutions, and chaos engineering practices is an advantage.

  • Strong problem‑solving, analytical, communication, and collaboration skills, with the ability to work effectively across multidisciplinary engineering teams.

Benefits
  • Fully remote working arrangement.

  • Full‑time employment opportunity.

  • Opportunity to work on large‑scale AI infrastructure and advanced generative AI technologies.

  • Exposure to distributed systems, GPU computing, and modern machine learning platforms.

  • Collaboration with experienced professionals across AI research, software development, and platform engineering.

  • Opportunities for continuous technical learning, innovation, and professional development.

  • The opportunity to contribute to mission‑critical AI services with global reach and impact.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Specialist in machine learning engineering
Specialist in machine learning engineering

AItoolnavio • India

Remote
INR 2,500,000 - 4,000,000
Fully remote
Senior Forward deployed Engineers
Senior Forward deployed Engineers

Jobgether • India

Remote
INR 2,500,000 - 4,000,000
Remote working in India
Exposure to LLMs / Gen AI
Enterprise cloud & data engineering
Senior Software Engineer
Senior Software Engineer

Clarus Advisors • Hyderabad

On-site
INR 2,800,000 - 4,000,000
Senior AI DevOps Engineer - Azure/AWS & Python
Senior AI DevOps Engineer - Azure/AWS & Python

NewVision • Pune District

Hybrid
INR 4,000,000 - 6,000,000
Hybrid work model
Pune office
AI-MLOps SRE Lead Engineer
AI-MLOps SRE Lead Engineer

Regeneron Pharmaceuticals, Inc. • Hyderabad

On-site
INR 3,500,000 - 7,000,000
Senior AI Engineer / Senior Machine Learning Specialist
Senior AI Engineer / Senior Machine Learning Specialist

Impronics Technologies • Gurugram District

On-site
INR 4,000,000 - 7,000,000
Technology Education Specialist
Technology Education Specialist

Jobgether • India

Remote
INR 1,800,000 - 2,600,000
Remote working: Fully remote in India
Professional development in AI/ML
Hands-on AI tech exposure
+6
Staff Machine Learning Engineer
Staff Machine Learning Engineer

Weekday AI • Bengaluru

On-site
INR 6,000,000 - 12,000,000
AI / ML Engineer
AI / ML Engineer

DataPhi • Pune District

On-site
INR 1,200,000 - 1,800,000
Senior AI ML Engineer
Senior AI ML Engineer

Impelsys • Bengaluru

On-site
INR 4,000,000 - 6,000,000