Lead Systems Engineer - Data DevOps MLOps

EPAM Systems; Hyderabad; On-site; INR 4,200,000 - 7,000,000; Full time

Job summary

EPAM Systems is seeking a Lead Systems Engineer with Data DevOps/MLOps expertise to advance our data and machine learning operations. You will build and oversee CI/CD pipelines for data integration and ML model rollout, and establish cloud-based infrastructure for training, processing, and serving models.

You will collaborate with data engineers and scientists to deploy ML solutions in production, improve reliability, monitoring, and model serving, and enforce data versioning, reproducibility,

Qualifications

• Bachelor's or Master's degree in Data Engineering, Computer Science, or related area.
• 8+ years in MLOps, Data DevOps, or similar fields.
• Strong command of cloud platforms (GCP/AWS/Azure).
• Proficiency with IaC tools like Ansible, CloudFormation, or Terraform.
• Experience with orchestration and containerization (Kubernetes, Docker).
• Experience with Databricks and Apache Spark.
• Strong Python skills with libraries like PyTorch, TensorFlow, Pandas.
• Familiarity with CI/CD tools (GitHub Actions, GitLab CI/CD, Jenkins).
• Knowledge of MLOps platforms (Kubeflow, MLflow) and Git.

Responsibilities

• Build, launch, and oversee CI/CD pipelines for data integration and ML model rollout.
• Create infrastructure foundation for training, processing, and serving ML models on cloud platforms.
• Automate workflows like data transformation, validation, and orchestration.
• Collaborate with data engineers and scientists to bring ML solutions into production.
• Enhance reliability, performance monitoring, and model serving in production systems.
• Maintain reproducibility, lineage tracking, and data versioning across ML workflows.
• Identify opportunities to improve resilience, efficiency, and scalability of infrastructure.
• Apply strict security protocols to protect data and ensure regulatory compliance.
• Troubleshoot and fix issues within ML deployment workflows and data pipelines.

Skills

Python programming

Data DevOps

MLOps

Problem solving

Technical documentation

Education

Bachelor's or Master's in Data Eng / CS

Tools

GCP

AWS

Azure

Kubernetes

Docker

Databricks

Apache Spark

GitHub Actions

GitLab CI/CD

Jenkins

Kubeflow

MLflow

Grafana

Prometheus

Job description

Job Description:

We're looking for a talented and dedicated Lead Systems Engineer who brings Data DevOps/MLOps expertise to help enhance efficiency and drive innovation in our data and machine learning operations.

Responsibilities

• Build, launch, and oversee CI/CD pipelines that enable smooth data integration and ML model rollout
• Create a strong infrastructure foundation for training, processing, and serving machine learning models through cloud-based platforms
• Streamline operations by automating essential workflows like data transformation, validation, and orchestration
• Partner with cross-functional teams such as data engineers and scientists to bring ML solutions into live production environments
• Enhance reliability, performance monitoring, and model serving within production systems
• Maintain reproducibility, lineage tracking, and data versioning throughout ML workflows and experiments
• Spot and act on opportunities to boost the infrastructure's resilience, efficiency, and scalability
• Apply strict security protocols to protect data and maintain regulatory compliance
• Troubleshoot and fix technical problems within ML deployment workflows and data pipelines

Requirements

• A Bachelor's or Master's degree in Data Engineering, Computer Science, or a related area
• Over 8 years working in MLOps, Data DevOps, or similar fields
• Strong command of cloud platforms including GCP, AWS, or Azure
• Proficiency with Infrastructure as Code tools such as Ansible, CloudFormation, or Terraform
• Capability in orchestration and containerization technologies like Kubernetes and Docker
• Practical experience using data processing frameworks such as Databricks and Apache Spark
• Strong Python skills along with familiarity with libraries like PyTorch, TensorFlow, and Pandas
• Familiarity with CI/CD tools including GitHub Actions, GitLab CI/CD, and Jenkins
• Working knowledge of MLOps platforms and version control systems such as Kubeflow, MLflow, and Git
• Awareness of alerting and monitoring tools such as Grafana and Prometheus
• Solid ability to solve problems and make decisions independently
• Strong skills in technical documentation and communication

Nice to have

• Experience with DataOps tools and methodologies such as dbt or Airflow
• Familiarity with data governance platforms such as Collibra
• Exposure to Big Data technologies like Hive or Hadoop
• Certifications demonstrating expertise in data engineering tools or cloud platforms