Senior Systems Engineer - Data DevOps MLOps

EPAM Systems; Hyderabad; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

EPAM Systems in Hyderabad, India is seeking a Senior Systems Engineer with deep Data DevOps/MLOps expertise to reinforce our team. You will design and deploy CI/CD pipelines for data integration and ML model deployment, and build cloud-based infrastructure to support data processing and model training.

You will automate data validation, versioning, and orchestration, collaborate with data scientists and software engineers, and enhance model serving, monitoring, and security across the ML

Qualifications

• Bachelor's or Master's in CS/Data Eng or related field.
• 5+ years in Data DevOps, MLOps, or similar roles.
• Proficient in Azure/AWS/GCP cloud platforms.
• Experience with IaC tools like Terraform, CloudFormation, or Ansible.
• Strong containerization with Docker and Kubernetes.
• Hands-on with Spark and Databricks.
• Proficient in Python; familiar with ML libraries such as Pandas, TensorFlow, PyTorch.
• Experience with Jenkins, GitLab CI/CD, GitHub Actions.
• Experience with Git, MLflow, Kubeflow.
• Knowledge of monitoring with Prometheus and Grafana.
• Strong communication and documentation skills.
• Ability to work independently and in teams.

Responsibilities

• Develop, deploy, and manage CI/CD pipelines for data integration and ML deployment.
• Set up and sustain infrastructure for data processing and model training through cloud-based resources and services.
• Automate processes for data validation, transformation, and workflow orchestration.
• Work closely with data scientists, software engineers, and product teams for a smooth integration of ML models into production.
• Enhance model serving and monitoring to boost performance and dependability.
• Manage data versioning, lineage tracking, and the reproducibility of ML experiments.
• Actively search for enhancements in deployment processes, scalability, and infrastructure resilience.
• Implement stringent security protocols to safeguard data integrity and compliance with regulations.
• Troubleshoot and solve issues throughout the data and ML pipeline lifecycle.

Skills

Problem-solving

Communication

Teamwork

Adaptability

Education

Bachelor's or Master's in CS/Data Eng or related field

Tools

Azure

AWS

GCP

Terraform

Ansible

Docker

Kubernetes

Apache Spark

Databricks

Python

Pandas

TensorFlow

PyTorch

Jenkins

GitLab CI/CD

GitHub Actions

MLflow

Kubeflow

Prometheus

Grafana

Git

Job description

Job Description:

We are looking for a dedicated and proficient Senior Systems Engineer with extensive Data DevOps/MLOps knowledge to enhance our team.

Responsibilities

• Develop, deploy, and manage Continuous Integration/Continuous Deployment (CI/CD) pipelines for data integration and machine learning model deployment
• Set up and sustain infrastructure for data processing and model training through cloud-based resources and services
• Automate processes for data validation, transformation, and workflow orchestration
• Work closely with data scientists, software engineers, and product teams for a smooth integration of ML models into production
• Enhance model serving and monitoring to boost performance and dependability
• Manage data versioning, lineage tracking, and the reproducibility of ML experiments
• Actively search for enhancements in deployment processes, scalability, and infrastructure resilience
• Implement stringent security protocols to safeguard data integrity and compliance with regulations
• Troubleshoot and solve issues throughout the data and ML pipeline lifecycle

Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field
• 5+ years of experience in Data DevOps, MLOps, or similar roles
• Proficiency in cloud platforms such as Azure, AWS, or GCP
• Background in Infrastructure as Code (IaC) tools like Terraform, CloudFormation, or Ansible
• Expertise in containerization and orchestration technologies including Docker and Kubernetes
• Hands-on experience with data processing frameworks such as Apache Spark and Databricks
• Proficiency in programming languages including Python with an understanding of data manipulation and ML libraries like Pandas, TensorFlow, and PyTorch
• Familiarity with CI/CD tools including Jenkins, GitLab CI/CD, and GitHub Actions
• Experience with version control tools and MLOps platforms such as Git, MLflow, and Kubeflow
• Strong understanding of monitoring, logging, and alerting systems including Prometheus and Grafana
• Excellent problem-solving abilities with capability to work independently and in teams
• Strong skills in communication and documentation

Nice to have

• Background in DataOps concepts and tools such as Airflow and dbt
• Knowledge of data governance platforms like Collibra
• Familiarity with Big Data technologies including Hadoop and Hive
• Certifications in cloud platforms or data engineering