Machine Learning Operations Engineer

EXL; Gurugram District, Dadri; On-site; INR 15,00,000 - 25,00,000; Full time

Job summary

A leading data and analytics firm in Gurugram is seeking an experienced MLOps Engineer to design, implement, and optimize data science pipelines. The candidate should have a strong background in Python, Spark/PySpark, and CI/CD practices. Responsibilities include managing infrastructure resources, collaborating on machine learning deployments, and automating ML workflows. The role requires excellent problem-solving skills and effective communication with stakeholders.

Qualifications

• 5+ years of prior experience in Data Engineering and MLOps.
• 3+ years of strong exposure in deploying and managing data science pipelines in production environments.
• Strong proficiency in Python programming language.
• Experience with Spark/PySpark and distributed computing frameworks.
• Hands-on experience with CI/CD pipelines and automation tools.
• Exposure in deploying a use case in production leveraging Generative AI.
• Familiarity with Kafka or similar messaging systems.
• Strong problem-solving skills.
• Ability to communicate effectively with diverse clients.

Responsibilities

• Design, develop, and maintain data science pipelines for model training.
• Manage and optimize infrastructure resources for model deployment.
• Collaborate with teams to deploy machine learning models.
• Automate end-to-end ML workflows.
• Implement CI/CD pipelines for automated model deployment.
• Utilize Kafka for real-time data processing.
• Optimize distributed computing infrastructure.
• Manage GitHub repositories for collaboration.

Skills

Spark/PySpark

MLOps

CI/CD

Kafka

Python

distributed computing

GitHub

data pipelines

cloud hosting

Azure services

Education

Bachelor's or master's degree in computer science or related field

Tools

Kubeflow

Apache Airflow

Job description

Job Description:

We are looking for an experienced MLOPs Engineer with expertise in Spark/PySpark, MLOps/LLMops/DLOps, CI/CD, Kafka, Python, distributed computing, GitHub, data pipelines, cloud hosting, Azure services, Microsoft services, various data connectors, and more. This role will involve designing, implementing, and optimizing data science pipelines, deploying machine learning models, and ensuring smooth operation in production environments.

Responsibilities:

• Design, develop, and maintain data science pipelines for model training, evaluation, and deployment.
• Manage and optimize infrastructure resources (e.g., cloud services, containers) to support model deployment and inference
• Collaborate with data scientists, software engineers, and DevOps teams to deploy machine learning models using best practices in MLOps.
• Automate end-to-end ML workflows, including data preprocessing, model training, evaluation, and deployment, using tools like Kubeflow or Apache Airflow
• Implement CI/CD pipelines for automated model deployment, testing, and monitoring.
• Utilize Kafka and other messaging systems for real-time data processing and streaming analytics.
• Optimize distributed computing infrastructure for scalability, performance, and cost efficiency.
• Manage GitHub repositories for version control and collaboration on machine learning projects.
• Utilize various data connectors and integration tools to access and process data from different sources.
• Develop and maintain documentation for data science pipelines, infrastructure, and processes.
• Stay up to date on emerging technologies and best practices in machine learning operations and data engineering.

Qualifications:

• 5+ Years of prior experience in Data Engineering and MLOPs.
• 3+ Years of strong exposure in deploying and managing data science pipelines in production environments.
• Strong proficiency in Python programming language.
• Experience with Spark/PySpark and distributed computing frameworks.
• Hands-on experience with CI/CD pipelines and automation tools.
• Exposure in deploying a use case in production leveraging Generative AI involving prompt engineering and RAG Framework
• Familiarity with Kafka or similar messaging systems.
• Strong problem-solving skills and the ability to iterate and experiment to optimize AI model behavior.
• Excellent problem-solving skills and attention to detail.
• Ability to communicate effectively with diverse clients/stakeholders.

Education Background:

• Bachelors or master’s degree in computer science, Engineering, or a related field.