Senior AI Engineer — Databricks ML Pipelines

EPAM Systems; Turkey; On-site; TRY 4,375,000 - 7,292,000; Full time

Benefits offered by this job

Private health insurance

Extra leave days

Referral bonuses

USD compensation

IT equipment

Job summary

EPAM Systems is seeking a Senior AI Engineer to design, deploy, and maintain scalable ML pipelines on Databricks. You will build end-to-end data workflows, automate training, and deploy models with monitoring and CI/CD for ML systems.

You will leverage PySpark, Spark ML, MLflow and Delta Lake, ensuring reproducibility and cost-efficient deployments while integrating data sources like ADLS and S3.

Qualifications

• 3+ years in machine learning engineering or related roles.
• Expertise with Databricks: workspaces, jobs and clusters.
• Proficient in PySpark, Python (pandas, scikit-learn).
• Experience with MLflow for tracking, registry and deployment.
• CI/CD, Docker, REST APIs for model serving, Git versioning.
• Azure Databricks, ADLS, ACR, AML experience.

Responsibilities

• Design, implement and maintain end-to-end ML pipelines on Databricks.
• Build data ingestion, preprocessing, feature engineering, training and inference workflows.
• Utilize PySpark, Spark ML and Databricks notebooks/jobs.
• Manage model versioning, experiments and reproducibility with MLflow.
• Package and deploy models for batch and real-time inference.
• Monitor performance, drift and retraining cycles; ensure observability.
• Develop scalable ETL/ELT pipelines with Delta Lake.
• Optimize storage, partitioning, and caching; cost-efficient design.
• Integrate data sources like ADLS, S3, APIs and databases.
• Configure CI/CD pipelines using Azure DevOps, GitHub Actions; Databricks Repos/Jobs API.
• Apply IaC with Terraform, ARM and Bicep; manage clusters and autoscaling.

Skills

Databricks

PySpark

Python

MLflow

CI/CD

Docker

Git

Azure Databricks

Azure Data Lake

MLOps

Tools

Terraform

ARM

Bicep

Databricks Repos

Job description

EPAM Systems is seeking a Senior AI Engineer to design, deploy, and maintain scalable ML pipelines on Databricks. You will build end-to-end data workflows, automate training, and deploy models with monitoring and CI/CD for ML systems.

You will leverage PySpark, Spark ML, MLflow and Delta Lake, ensuring reproducibility and cost-efficient deployments while integrating data sources like ADLS and S3.