Lead Systems Engineer - Data DevOps/MLOps

EPAM Systems; Coimbatore District; In loco; INR 3.500.000 - 7.000.000; Tempo pieno

Descrizione del lavoro

EPAM Systems in Coimbatore is seeking a Lead Systems Engineer with Data DevOps and MLOps expertise to enhance data and ML operations. You will build and oversee CI/CD pipelines, create scalable cloud infrastructure for training and serving models, and automate critical workflows.

Ideal candidates have 8+ years in MLOps, strong Python skills, and hands-on experience with Kubernetes, Kubeflow, Databricks, and cloud platforms.

Competenze

• Bachelor's or Master's in data engineering, CS, or related field.
• 8+ years in MLOps/Data DevOps or similar.
• Strong cloud experience (GCP/AWS/Azure).
• Proficient with IaC tools such as Terraform or CloudFormation.

Mansioni

• Build and manage CI/CD pipelines for data integration and ML model rollout.
• Create infrastructure for training, processing, and serving ML models on cloud platforms.
• Automate data transformation, validation, and orchestration.
• Collaborate with data engineers and scientists to deploy ML solutions in production.
• Improve reliability and monitoring of production systems.
• Maintain reproducibility, lineage, and data versioning in ML workflows.
• Identify opportunities to improve infrastructure resilience and scalability.
• Apply security protocols to protect data and ensure compliance.
• Troubleshoot issues in ML deployment workflows and data pipelines.
• Experience with DataOps tools like dbt or Airflow.
• Familiarity with data governance tools like Collibra.
• Exposure to Hive or Hadoop.

Conoscenze

MLOps

Data DevOps

Python

Kubernetes

Docker

Databricks

Apache Spark

CI/CD

Cloud Platforms

Formazione

Bachelor's or Master's in CS/Data Engineering

Strumenti

Kubeflow

MLflow

GitHub Actions

GitLab CI/CD

Jenkins

Terraform

Ansible

Descrizione del lavoro

We're looking for a talented and dedicated Lead Systems Engineer who brings Data DevOps/MLOps expertise to help enhance efficiency and drive innovation in our data and machine learning operations.

Responsibilities

• Build, launch, and oversee CI/CD pipelines that enable smooth data integration and ML model rollout
• Create a strong infrastructure foundation for training, processing, and serving machine learning models through cloud-based platforms
• Streamline operations by automating essential workflows like data transformation, validation, and orchestration
• Partner with cross-functional teams such as data engineers and scientists to bring ML solutions into live production environments
• Enhance reliability, performance monitoring, and model serving within production systems
• Maintain reproducibility, lineage tracking, and data versioning throughout ML workflows and experiments
• Spot and act on opportunities to boost the infrastructure's resilience, efficiency, and scalability
• Apply strict security protocols to protect data and maintain regulatory compliance
• Troubleshoot and fix technical problems within ML deployment workflows and data pipelines

Requirements

• A Bachelor's or Master's degree in Data Engineering, Computer Science, or a related area
• Over 8 years working in MLOps, Data DevOps, or similar fields
• Strong command of cloud platforms including GCP, AWS, or Azure
• Proficiency with Infrastructure as Code tools such as Ansible, CloudFormation, or Terraform
• Capability in orchestration and containerization technologies like Kubernetes and Docker
• Practical experience using data processing frameworks such as Databricks and Apache Spark
• Strong Python skills along with familiarity with libraries like PyTorch, TensorFlow, and Pandas
• Familiarity with CI/CD tools including GitHub Actions, GitLab CI/CD, and Jenkins
• Working knowledge of MLOps platforms and version control systems such as Kubeflow, MLflow, and Git
• Awareness of alerting and monitoring tools such as Grafana and Prometheus
• Solid ability to solve problems and make decisions independently
• Strong skills in technical documentation and communication

Nice to have

• Experience with DataOps tools and methodologies such as dbt or Airflow
• Familiarity with data governance platforms such as Collibra
• Exposure to Big Data technologies like Hive or Hadoop
• Certifications demonstrating expertise in data engineering tools or cloud platforms