Lead AI Solution Engineer - Databricks & Real-Time ML

EPAM Systems Inc; United States; Remote; USD 150,000 - 230,000; Full time

Job summary

EPAM Systems Inc. is seeking a Lead AI Solution Engineer in the United States who combines deep Databricks expertise with ML to architect, launch, and operate production-grade ML systems. The role blends hands-on data engineering and ML engineering to deliver real-time and batch solutions.

You will design streaming pipelines with Spark Streaming, optimize feature engineering, and ensure low-latency inference while maintaining reliability and scalability across production environments.

Qualifications

• 7+ years of experience building, deploying, and operating offline and online feature engineering pipelines.
• Proficiency in Databricks, Spark/PySpark, Python, and SQL.
• Expertise in model training and real-time inferencing pipelines.
• Experience processing large-scale unstructured data.
• Knowledge of Spark Streaming.
• Experience with real-time feature generation and low-latency model inference.
• Proven track record of running production-grade ML systems.
• English proficiency at B2 level or higher.
• Nice to have Exposure to MLflow and established MLOps workflows.
• Hands-on background with AWS.
• Prior work in Customer Analytics, Recommendation Systems, or Retail industries.

Responsibilities

• Design and optimize feature engineering pipelines for both batch and real-time workloads
• Build, deploy, and maintain scalable model training and inference pipelines
• Process structured and unstructured data at scale
• Develop streaming solutions using Spark Streaming
• Enable real-time feature generation and model serving
• Ensure reliability, scalability, and performance of ML solutions in production
• Support batch pipelines for model training and retraining
• Operate and maintain production ML systems to ensure consistent performance

Skills

Databricks

Spark/PySpark

Python

SQL

Real-time feature generation

Low-latency inference

Production ML systems

AWS

Unstructured data processing

MLOps (MLflow)

Spark Streaming

Customer Analytics / Retail domain

Tools

MLflow

AWS

Spark Streaming

Spark

Job description

EPAM Systems Inc. is seeking a Lead AI Solution Engineer in the United States who combines deep Databricks expertise with ML to architect, launch, and operate production-grade ML systems. The role blends hands-on data engineering and ML engineering to deliver real-time and batch solutions.

You will design streaming pipelines with Spark Streaming, optimize feature engineering, and ensure low-latency inference while maintaining reliability and scalability across production environments.