Senior MLOps or Real-Time ML Engineer

eClerx LLC; Pune District, Mumbai; Presencial; INR 4.200.000 - 6.500.000; Jornada completa

Descripción de la vacante

eClerx LLC is seeking a Senior MLOps / Real-Time ML Engineer with 8+ years of hands-on Databricks experience to build, productionize and operate enterprise-scale ML solutions.

The role combines MLOps, ML engineering and real-time data engineering, spanning model development, deployment, monitoring and optimization across the full ML lifecycle. You will design real-time pipelines, implement CI/CD for ML models and collaborate with data science, data engineering and platform teams.

Formación

• 8+ years of experience in MLOps, ML engineering or related roles.

Responsabilidades

• Design and productionize ML/MLOps solutions on Databricks.
• Build real-time ML pipelines using Kafka and Databricks Structured Streaming.
• Develop offline and online features with training-serving parity.
• Productionize models using MLflow, Unity Catalog, and Databricks Model Serving.
• Build and automate CI/CD pipelines for ML models and data/ML workflows.
• Design low-latency inference and real-time decisioning solutions.
• Implement monitoring for model performance, data drift, latency, and cost.
• Optimize Spark/Databricks workloads for scalability and performance.
• Collaborate with DS, DE, Platform and Product teams to define scalable ML architecture.

Conocimientos

Databricks

Python

PySpark

SQL

Kafka

Spark Streaming

MLflow

Delta Lake

Model Serving

CI/CD

Cloud Platforms

XGBoost

LightGBM

scikit-learn

Docker

Kubernetes

Herramientas

Databricks

MLflow

Unity Catalog

Delta Lake

Databricks Workflows

Lakeflow

Kafka

Spark Structured Streaming

XGBoost

LightGBM

scikit-learn

TensorFlow

Terraform

Docker

Kubernetes

Descripción del empleo

Senior MLOps / Real-Time ML Engineer Databricks

Experience: 8+ years

Role Overview

We are looking for a Senior MLOps / Real-Time ML Engineer with 8+ years of experience and extensive hands‑on experience with Databricks to build, productionize and operate enterprise-scale ML solutions.

The role combines MLOps, ML engineering and real‑time/streaming data engineering, with ownership across the ML lifecycle—from model development and feature engineering to deployment, monitoring and optimization.

Key Responsibilities

• Design and productionize scalable ML/MLOps solutions on Databricks, covering model training, deployment, monitoring, retraining and lifecycle management.
• Build real‑time ML pipelines using Kafka and Databricks Structured Streaming, including stateful processing, event‑time logic and real‑time feature generation.
• Develop offline and online features with strong focus on point‑in‑time correctness, leakage prevention and training‑serving parity.
• Productionize models using MLflow, Unity Catalog, Databricks Model Serving and Databricks‑native capabilities.
• Build and automate CI/CD pipelines for ML models and data/ML workflows across development, testing and production.
• Design low‑latency inference and real‑time decisioning solutions capable of continuously processing live events and generating predictions.
• Implement monitoring for model performance, data/feature drift, pipeline health, latency, reliability and cost.
• Optimize Spark/Databricks workloads for scalability, performance and cost.
• Partner with Data Science, Data Engineering, Platform and Product teams to define and implement scalable ML architecture.
• Build reusable MLOps and real‑time ML frameworks that can be applied across multiple use cases.

Required Skills & Experience

• 8+ years in MLOps, ML Engineering, Data Science, Data Engineering or Software Engineering, with significant production ML experience.
• Extensive hands‑on Databricks experience, including MLflow, Unity Catalog, Delta Lake and Databricks Workflows/Lakeflow Jobs.
• Strong Python, PySpark and SQL skills.
• Hands‑on experience with Kafka and Spark Structured Streaming, including stateful/event‑time processing.
• Experience building real‑time ML inference and decisioning pipelines.
• Strong understanding of feature engineering, offline/online features, training‑serving parity and data leakage prevention.
• Experience with ML frameworks such as XGBoost, LightGBM or scikit‑learn.
• Strong understanding of classification, regression, time‑series/event‑based and/or time‑to‑event prediction.
• Experience with model deployment, monitoring, model lifecycle management and CI/CD.
• Experience with cloud platforms such as Azure, AWS or GCP; Azure Databricks experience is desirable.
• Ability to make and communicate architecture decisions across ML, data and platform layers.

Good to Have

• Databricks Feature Engineering / Feature Store and online feature stores.
• Databricks Model Serving and low‑latency inference platforms.
• transformWithState or equivalent stateful streaming frameworks.
• Experience in real‑time decisioning, logistics, supply chain, fraud or recommendation systems.
• Survival analysis / time‑to‑event modeling.
• Terraform, Docker and Kubernetes.
• Experience with GenAI/LLMOps.

What We Are Looking For

A hands‑on engineer who can take an ML solution from prototype to production, build the real‑time data and feature pipelines around it, deploy and monitor the model, and establish reusable MLOps capabilities for reliable ML at scale.