Lead ML/AI Solution Engineer with Databricks

EPAM Systems

United States

Remote

USD 180,000 - 240,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

EPAM Systems Inc. seeks a seasoned Lead ML/AI Solution Engineer to architect and launch a personalized recommendation platform. You will work with Data Scientists to operationalize models via a scalable FastAPI API, backed by Databricks Model Serving, and managed on AWS with Terraform-driven infra.

You will lead production-grade engineering, implement caching and observability, and ensure multi-region readiness with OpenAPI-first design and contract testing.

Qualifications

  • 5+ years of ML/AI engineering experience, building and serving recommendation or ranking models.

Responsibilities

  • Partner with Data Scientists to adapt, package, and deploy personalization models to Databricks Model Serving.
  • Develop and maintain a FastAPI service that invokes model serving endpoints.
  • Apply production-grade engineering practices including caching, connection pooling, timeouts, and fail-open fallbacks.
  • Set up secure authentication between services and Databricks using Service Principals.
  • Establish and maintain the AWS integration layer with API Gateway and Lambda.
  • Define infrastructure with Terraform and build a CI/CD pipeline.
  • Enable observability and monitoring, including latency, error, and fallback metrics.
  • Run load and performance testing to validate system behavior under traffic.
  • Specify REST API contracts and perform contract testing (Pact).
  • Carry out offline evaluation and online validation via A/B/shadow testing where feasible.
  • Present technical results and business impact to stakeholders.

Skills

Python
FastAPI
MLflow
Databricks
OpenAPI
Contract testing
Observability
Load testing
AWS
Service Principals

Tools

Terraform
AWS API Gateway
AWS Lambda
K6/Locust
Pact
Databricks Model Serving
MLflow

Job description

We are seeking a seasoned Lead ML/AI Solution Engineer to architect, develop, and launch a personalization platform that delivers recommendation and ranking models via a scalable API layer. The system comprises a Python FastAPI service exposed through AWS API Gateway and Lambda, integrating with Databricks Model Serving endpoints. The engineer will partner closely with Data Scientists to operationalize personalization models, guarantee production-grade reliability, and confirm performance through testing and monitoring.

Responsibilities
  • Partner with Data Scientists to adapt, package, and deploy personalization models (recommendation and ranking models) to Databricks Model Serving using MLflow
  • Develop and maintain a FastAPI service that invokes one or more model serving endpoints
  • Apply production-grade engineering practices, including caching, connection pooling, request timeouts, and fail-open fallback mechanisms
  • Set up secure authentication between services and Databricks using Service Principals
  • Establish and maintain the AWS integration layer, including API Gateway, AWS Lambda, and networking/egress connectivity
  • Define and govern infrastructure using Terraform and build a simple CI/CD pipeline
  • Enable observability and monitoring, including latency, error, and fallback metrics
  • Run load and performance testing to validate system behavior under expected traffic volumes
  • Specify REST API contracts and implement contract testing to enable early integration by consuming teams
  • Carry out offline evaluation of personalization models and support online validation through A/B testing or shadow testing where feasible
  • Present technical results and business impact to both technical and non-technical stakeholders
Requirements
  • 5+ years of experience in ML/AI engineering, building and serving recommendation or ranking models, including offline model evaluation
  • At least 1 year of relevant leadership experience
  • Hands-on knowledge of Collaborative Filtering, Two-Tower Architectures, and Learning-to-Rank approaches
  • Strong proficiency in Python with production experience building services using FastAPI
  • Understanding of asynchronous processing, connection pooling, and caching strategies
  • Expertise in timeout handling, fail-open mechanisms, and resilient service design
  • Background in registering, versioning, and serving ML models with MLflow and Databricks Model Serving
  • Familiarity with Service Principal authentication, endpoint scaling, and version management
  • Competency in monitoring latency, errors, and fallback rates, including creating alerts and dashboards
  • Skills in load-testing tools such as k6 or Locust, with the capability to validate p99 latency targets
  • Capability to explain technical concepts and outcomes in business terms, working with engineering, data science, and business stakeholders
  • English proficiency at an Upper-Intermediate level (B2) or higher
  • Nice to have Proficiency in Infrastructure-as-Code with Terraform and CI/CD pipeline implementation adequate to deploy and hand over a reproducible POC environment
  • Familiarity with AWS API Gateway and Lambda integration, including awareness of cold starts and performance optimization
  • Knowledge of VPC networking and egress connectivity to Databricks
  • Expertise in OpenAPI-first REST API design and contract testing with Pact or schema-based testing approaches
  • Capability to architect services for multi-region deployment and future scalability
  • Understanding of GenAI for personalization, including embeddings, vector search, LLM-based re-ranking, and personalized explanations powered by GenAI
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead ML/AI Solution Engineer with Databricks
Lead ML/AI Solution Engineer with Databricks

EPAM Systems Inc • Newtown (PA)

On-site
USD 140,000 - 190,000
Career development
Private health insurance
Referral bonuses
+1
Lead ML/AI Engineer for Personalization & Databricks
Lead ML/AI Engineer for Personalization & Databricks

EPAM Systems • United States

Remote
USD 180,000 - 240,000
Senior ML/AI Engineer
Senior ML/AI Engineer

GuruLink • Nashville (TN)

Remote
USD 140,000 - 210,000
Lead ML/AI Engineer
Lead ML/AI Engineer

GuruLink • Nashville (TN)

Remote
USD 180,000 - 240,000
Machine Learning Engineer, Personalization
Machine Learning Engineer, Personalization

Bask Health • New York (NY)

On-site
USD 120,000 - 190,000
Senior AI/ML Platform Engineer
Senior AI/ML Platform Engineer

Employee Relations Associates, Inc. • Denver (CO)

Hybrid
USD 170,000 - 210,000
Medical coverage
Dental coverage
Vision coverage
+4
Machine Learning Engineer, Personalization
Machine Learning Engineer, Personalization

Bask Health • United States

Remote
USD 150,000 - 210,000
AI/ML Architect with Databricks , AWS
AI/ML Architect with Databricks , AWS

Vytwo Technologies Inc. • Prosper (TX)

Remote
USD 180,000 - 240,000
Remote work options
ML Lead
ML Lead

Cyberobotix • Atlanta (GA)

Hybrid
USD 170,000 - 260,000
Senior AI/ML Engineer | Data Science
Senior AI/ML Engineer | Data Science

Tech in Motion • Dallas (TX)

On-site
USD 140,000 - 190,000