Senior AI Engineer with Databricks

EPAM Systems; Turkey; On-site; TRY 250,000 - 450,000; Full time

Benefits offered by this job

Private health insurance

Continuous upskilling

English courses

USD compensation package

Learning platforms access

Job summary

EPAM Systems is seeking a Senior AI Engineer to design, deploy and maintain scalable ML pipelines on the Databricks platform. You will deliver production-ready pipelines, automated training and retraining workflows, and CI/CD for ML systems.

The role requires Databricks expertise, PySpark, Python, MLflow, and experience with Azure Databricks, ADLS and Terraform. Strong communication in English is expected in a global team.

Qualifications

• 3+ years of ML engineering experience with end-to-end pipelines.
• Databricks platform expertise including notebooks, jobs and clusters.
• Proficient in Spark, PySpark, Python (pandas, scikit-learn).
• MLflow for tracking, registry and deployment; CI/CD for ML.

Responsibilities

• Design, build and maintain end-to-end ML pipelines on Databricks.
• Create data ingestion, preprocessing, feature engineering and training workflows.
• Develop batch and real-time inference deployments with monitoring.
• Implement CI/CD pipelines for ML workflows using Azure DevOps and GitHub Actions.
• Manage model versioning, experiment tracking and reproducibility with MLflow.
• Optimize storage, partitioning, and cost using Terraform, ARM and Bicep.
• Provide logging, alerts and observability for high availability.

Skills

Databricks

PySpark

Python

Pandas

Scikit-learn

MLflow

CI/CD

Docker

REST APIs

Git

Azure Databricks

Azure Data Lake

Terraform

Monitoring

Education

Bachelor's degree in CS/Math/Engineering

Master's degree preferred

Tools

Azure DevOps

GitHub Actions

Databricks Repos

Terraform

ARM

Bicep

Job description

We are looking for a Senior AI Engineer with Databricks expertise to design, deploy and maintain scalable machine learning pipelines using the Databricks platform. In this role, you will deliver production-ready ML pipelines, automated training and retraining workflows, deployed models, monitoring dashboards and CI/CD pipelines for ML systems.

Responsibilities

• Design, implement and maintain end-to-end ML pipelines on Databricks
• Build workflows for data ingestion, preprocessing, feature engineering, training and inference
• Leverage PySpark, Spark ML and Databricks notebooks/jobs
• Manage model versioning, experiment tracking and reproducibility using MLflow
• Package and deploy models for batch and real-time inference
• Monitor model performance, drift and retraining cycles
• Optimize data storage and access patterns through partitioning, Z-ordering and caching
• Integrate with data sources such as Azure Data Lake, S3, APIs and databases
• Implement CI/CD pipelines for ML workflows using Azure DevOps, GitHub Actions and Databricks Repos and Jobs API
• Configure clusters, autoscaling and cost optimization while applying Infrastructure as Code with Terraform, ARM and Bicep
• Implement logging, alerting and observability to ensure high availability and fault tolerance of ML systems

Requirements

• 3+ years of experience in machine learning engineering or related roles
• Expertise in the Databricks platform including workspaces, jobs and clusters
• Proficiency in Apache Spark, PySpark and Python with pandas and scikit-learn
• Skills in MLflow for tracking, registry and deployment
• Competency in CI/CD pipelines, Docker containerization and REST APIs for model serving
• Familiarity with version control using Git
• Background in Azure including Azure Databricks, ADLS, ACR and AML
• Knowledge of data preprocessing, feature engineering and model training and evaluation
• Understanding of libraries such as XGBoost, LightGBM and CatBoost
• English proficiency at B2 level or higher

Nice to have

• Familiarity with AWS including S3, EMR and SageMaker
• Skills in streaming pipelines with Spark Structured Streaming and Databricks Feature Store
• Knowledge of Kubernetes
• Competency in monitoring tools such as Prometheus and Grafana
• Experience with large-scale production systems

We offer

• CONTINUOUS UPSKILLING, LEARNING & DEVELOPMENT
  • Diversity of tasks and projects
  • Assessment center for objective review of competency level
  • Personal development plan
  • Mentoring programs and leadership development
  • Certification and professional development support
  • Access to learning platforms including more than 2,500 internal courses
  • English courses taught by certified teachers
• CORPORATE BENEFITS
  • Extra leave days
  • Referral bonuses
• COMPENSATION PACKAGE
  • Competitive compensation paid in USD
  • Regular salary and performance reviews
• MEDICAL & HEALTHCARE
  • Private health insurance
  • Well-being events
• WORKING ENVIRONMENT
  • Recreation areas and kitchens
  • Tea, coffee and snacks
  • Sports equipment and game consoles
  • IT Equipment
  • Microsoft's Software Assurance Home Use Program (HUP)

Please note that our Talent Attraction Team reviews applications and CVs submitted in English.

EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.