Senior Data Engineer

EPAM Systems; Bogotá ciudad; Vor Ort; COP 120.000.000 - 240.000.000; Vollzeit

Benefits dieser Stelle

Healthcare benefits

LinkedIn Learning access

Global career opportunities

Zusammenfassung

EPAM Systems in Bogotá seeks a Senior Data Engineer to build an MLOps platform, enabling automated model delivery and scalable feature engineering. You will evolve the featureOps stack using Azure Databricks, Azure Data Factory, and Azure DevOps to run CI/CD into the Databricks Feature Store for Model Lab use cases.

Responsibilities include migrating Spark pipelines to Azure, collaborating on forecasting models, and bringing ML models into production while ensuring reliable, scalable data

Qualifikationen

• Hands-on experience with PySpark
• Solid Data Science core skills
• Proven track record in feature engineering and ML pipelines
• Practical experience migrating data pipelines to cloud environments
• English proficiency at B2 level or higher

Aufgaben

• Migrate Spark feature engineering pipelines to Azure
• Define and build a forecasting model with the team
• Take developed ML models into production
• Design and implement CI/CD for Databricks Feature Store
• Collaborate with Data Scientists and engineers on feature needs
• Maintain reliability and scalability of data pipelines
• Diagnose and improve Spark-based workflows during migration

Kenntnisse

PySpark

Data Science

Feature engineering

ML pipelines

Cloud migrations

Tools

MLflow

Docker

Azure Kubernetes Services

Azure Data Factory

Azure Databricks

Jobbeschreibung

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

We are seeking a Senior Data Engineer to build an MLOps platform and enable automated model delivery, deployment, and feature engineering at scale. You will evolve the featureOps stack using Azure Databricks, Azure Data Factory, and Azure DevOps to run CI/CD into the Databricks Feature Store for Model Lab use cases.

Responsibilities

• Migrate current Spark feature engineering pipelines to Azure
• Partner with the team to define and build a forecasting model
• Assist with taking the developed machine learning model into production
• Help design and implement the CI/CD pipeline that feeds the Databricks Feature Store
• Collaborate closely with Data Scientists and engineers to match feature engineering to ML model needs
• Maintain reliability and scalability of data pipelines across the platform
• Diagnose and improve existing Spark-based workflows throughout the migration

Requirements

• Hands-on experience of 3+ years working with PySpark
• Solid background in Data Science core skills
• Proven track record in feature engineering and machine learning pipeline development
• Practical experience migrating data pipelines to cloud environments
• English proficiency at B2 (Upper-Intermediate) level or higher

Nice to have

• Experience with MLflow and Docker
• Working knowledge of Azure Kubernetes Services, Azure Data Factory, and Azure Databricks
• Familiarity with ML model deployment practices

We offer

• International projects with top brands
• Work with global teams of highly skilled, diverse peers
• Healthcare benefits
• Employee financial programs
• Paid time off and sick leave
• Upskilling, reskilling and certification courses
• Unlimited access to the LinkedIn Learning library and 22,000+ courses
• Global career opportunities
• Volunteer and community involvement opportunities
• EPAM Employee Groups
• Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn

EPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.