MLOps Data Engineer | Azure Databricks & Feature Store

EPAM Systems; Bogotá ciudad; On-site; COP 12,000,000 - 18,000,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Upskilling & certification courses

Global career opportunities

Job summary

EPAM Systems is seeking a Data Engineer to join a team developing an MLOps platform that automates machine learning model development, deployment, and feature engineering. The platform leverages Azure Databricks, Azure Data Factory, and Azure DevOps to build a CI/CD pipeline that feeds the Databricks Feature Store, enabling ML model development in Model Lab.

The role involves migrating Spark pipelines to Azure, collaborating on forecasting models, and deploying ML solutions to production, while

Qualifications

• 2+ years of experience with PySpark.
• Strong background in Data Science core skills.
• Proficiency in feature engineering and ML pipeline development.
• Experience migrating data pipelines to cloud environments.
• English proficiency at B2 level or higher.

Responsibilities

• Migrate Spark feature engineering pipelines to Azure.
• Collaborate to identify and develop a forecasting model.
• Support deployment of ML model to production.
• Design and develop CI/CD pipeline feeding the Databricks Feature Store.
• Work with Data Scientists and engineers to align feature engineering with ML requirements.
• Ensure reliability and scalability of data pipelines across the platform.
• Troubleshoot and optimize Spark-based workflows during migration.

Skills

PySpark

Data science

ML pipeline development

Cloud migration

English proficiency

Tools

Azure Databricks

Azure Data Factory

Azure DevOps

MLflow

Docker

Kubernetes

Job description

EPAM Systems is seeking a Data Engineer to join a team developing an MLOps platform that automates machine learning model development, deployment, and feature engineering. The platform leverages Azure Databricks, Azure Data Factory, and Azure DevOps to build a CI/CD pipeline that feeds the Databricks Feature Store, enabling ML model development in Model Lab.

The role involves migrating Spark pipelines to Azure, collaborating on forecasting models, and deploying ML solutions to production, while