Lead Systems Engineer - Data DevOps/MLOps

Epam Systems; Coimbatore District; On-site; INR 3,500,000 - 5,200,000; Full time

Job summary

Epam Systems is seeking a Lead Systems Engineer with Data DevOps/MLOps expertise to drive innovation across data and ML operations. You will design and manage CI/CD pipelines for data integration, and deploy ML models using cloud-based infrastructure.

The role involves collaborating with data scientists and engineers, ensuring scalable, secure, and reliable ML deployments with robust data versioning and governance. Strong Python, Spark, and cloud skills are essential.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
• 8+ years of experience in Data DevOps, MLOps, or related disciplines.
• Expertise in cloud platforms such as Azure, AWS, or GCP.
• Skills in Infrastructure as Code tools like Terraform, CloudFormation, or Ansible.
• Proficiency in containerization and orchestration technologies such as Docker and Kubernetes.
• Hands-on experience with data processing frameworks including Apache Spark and Databricks.
• Proficiency in Python with familiarity with libraries including Pandas, TensorFlow, and PyTorch.
• Knowledge of CI/CD tools such as Jenkins, GitLab CI/CD, and GitHub Actions.
• Experience with version control systems and MLOps platforms including Git, MLflow, and Kubeflow.
• Understanding of monitoring and alerting tools like Prometheus and Grafana.
• Strong problem-solving and independent decision-making capabilities.
• Effective communication and technical documentation skills.

Responsibilities

• Design, deploy, and manage CI/CD pipelines for data integration and ML model deployment.
• Establish infrastructure for processing, training, and serving ML models using cloud-based solutions.
• Automate data validation, transformation, and orchestration workflows.
• Collaborate with data scientists and engineers to deploy ML solutions to production.
• Improve model serving and monitoring for reliability in production.
• Ensure data versioning, lineage tracking, and reproducibility across ML experiments.
• Identify opportunities to improve scalability, efficiency, and resilience of the infrastructure.
• Enforce security measures to safeguard data and ensure regulatory compliance.
• Debug and resolve issues in data pipelines and ML deployment workflows.

Skills

Problem solving

Independent decision making

Communication

Documentation

Education

Bachelor's or Master's in CS/Data Engineering

Tools

Terraform

CloudFormation

Ansible

Docker

Kubernetes

Apache Spark

Databricks

Python (Pandas, TF, PyTorch)

Jenkins

GitLab CI/CD

GitHub Actions

Git

MLflow

Kubeflow

Prometheus

Grafana

Job description

We are seeking a skilled and passionate Lead Systems Engineer with Data DevOps/MLOps expertise to drive innovation and efficiency across our data and machine learning operations.

Responsibilities

• Design, deploy, and manage CI/CD pipelines for seamless data integration and ML model deployment


• Establish robust infrastructure for processing, training, and serving machine learning models using cloud-based solutions


• Automate critical workflows such as data validation, transformation, and orchestration for streamlined operations


• Collaborate with cross-functional teams, including data scientists and engineers, to integrate ML solutions into production environments


• Improve model serving, performance monitoring, and reliability in production ecosystems


• Ensure data versioning, lineage tracking, and reproducibility across ML experiments and workflows


• Identify and implement opportunities to improve scalability, efficiency, and resilience of the infrastructure


• Enforce rigorous security measures to safeguard data and ensure compliance with relevant regulations


• Debug and resolve technical issues in data pipelines and ML deployment workflows

Requirements

• Bachelors or Masters degree in Computer Science, Data Engineering, or a related field


• 8+ years of experience in Data DevOps, MLOps, or related disciplines


• Expertise in cloud platforms such as Azure, AWS, or GCP


• Skills in Infrastructure as Code tools like Terraform, CloudFormation, or Ansible


• Proficiency in containerization and orchestration technologies such as Docker and Kubernetes


• Hands-on experience with data processing frameworks including Apache Spark and Databricks


• Proficiency in Python with familiarity with libraries including Pandas, TensorFlow, and PyTorch


• Knowledge of CI/CD tools such as Jenkins, GitLab CI/CD, and GitHub Actions


• Experience with version control systems and MLOps platforms including Git, MLflow, and Kubeflow


• Understanding of monitoring and alerting tools like Prometheus and Grafana


• Strong problem-solving and independent decision-making capabilities


• Effective communication and technical documentation skills

Nice to have

• Background in DataOps methodologies and tools such as Airflow or dbt


• Knowledge of data governance platforms like Collibra


• Familiarity with Big Data technologies such as Hadoop or Hive


• Showcase of certifications in cloud platforms or data engineering tools