Lead ML/AI Solution Engineer with Databricks

EPAM Systems Inc

Newtown (PA)

On-site

USD 140,000 - 190,000

Full time

9 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Career development
Private health insurance
Referral bonuses
English courses

Job summary

EPAM Systems Inc. seeks a seasoned Lead ML/AI Solution Engineer to architect and launch a personalization platform delivering recommendation and ranking models via a scalable API layer.

You will partner with data scientists to operationalize models on Databricks Model Serving and build a robust FastAPI service on AWS, ensuring reliability and performance under load.

Qualifications

  • 5+ years in ML/AI engineering with model serving experience.
  • Leadership experience and ability to drive cross-functional work.
  • Python production services with FastAPI and cloud deployment.
  • Experience with collaborative filtering, two-tower, and learning-to-rank.
  • MLflow, Databricks Model Serving, and model versioning knowledge.

Responsibilities

  • Architect, develop and deploy personalization models (rec/s ranking).
  • Build and maintain a FastAPI service calling model endpoints.
  • Establish caching, pooling, timeouts, and fail-open fallbacks.
  • Set up Service Principal authentication and AWS integration.
  • Define Terraform infra and CI/CD for reproducible POC.
  • Instrument observability and run performance tests.
  • Contract test REST APIs for early team integration.

Skills

Leadership experience
Stakeholder communication
Problem solving
Team collaboration

Tools

Python
FastAPI
Databricks
MLflow
AWS API Gateway
AWS Lambda
Terraform
CI/CD
OpenAPI
k6/Locust

Job description

We are seeking a seasoned Lead ML/AI Solution Engineer to architect, develop, and launch a personalization platform that delivers recommendation and ranking models via a scalable API layer. The system comprises a Python FastAPI service exposed through AWS API Gateway and Lambda, integrating with Databricks Model Serving endpoints.The engineer will partner closely with Data Scientists to operationalize personalization models, guarantee production-grade reliability, and confirm performance through testing and monitoring.ResponsibilitiesPartner with Data Scientists to adapt, package, and deploy personalization models (recommendation and ranking models) to Databricks Model Serving using MLflowDevelop and maintain a FastAPI service that invokes one or more model serving endpointsApply production-grade engineering practices, including caching, connection pooling, request timeouts, and fail-open fallback mechanismsSet up secure authentication between services and Databricks using Service PrincipalsEstablish and maintain the AWS integration layer, including API Gateway, AWS Lambda, and networking/egress connectivityDefine and govern infrastructure using Terraform and build a simple CI/CD pipelineEnable observability and monitoring, including latency, error, and fallback metricsRun load and performance testing to validate system behavior under expected traffic volumesSpecify REST API contracts and implement contract testing to enable early integration by consuming teamsCarry out offline evaluation of personalization models and support online validation through A/B testing or shadow testing where feasiblePresent technical results and business impact to both technical and non-technical stakeholdersRequirements5+ years of experience in ML/AI engineering, building and serving recommendation or ranking models, including offline model evaluationAt least 1 year of relevant leadership experienceHands-on knowledge of Collaborative Filtering, Two-Tower Architectures, and Learning-to-Rank approachesStrong proficiency in Python with production experience building services using FastAPIUnderstanding of asynchronous processing, connection pooling, and caching strategiesExpertise in timeout handling, fail-open mechanisms, and resilient service designBackground in registering, versioning, and serving ML models with MLflow and Databricks Model ServingFamiliarity with Service Principal authentication, endpoint scaling, and version managementCompetency in monitoring latency, errors, and fallback rates, including creating alerts and dashboardsSkills in load-testing tools such as k6 or Locust, with the capability to validate p99 latency targetsCapability to explain technical concepts and outcomes in business terms, working with engineering, data science, and business stakeholdersEnglish proficiency at an Upper-Intermediate level (B2) or higherNice to haveProficiency in Infrastructure-as-Code with Terraform and CI/CD pipeline implementation adequate to deploy and hand over a reproducible POC environmentFamiliarity with AWS API Gateway and Lambda integration, including awareness of cold starts and performance optimizationKnowledge of VPC networking and egress connectivity to DatabricksExpertise in OpenAPI-first REST API design and contract testing with Pact or schema-based testing approachesCapability to architect services for multi-region deployment and future scalabilityUnderstanding of GenAI for personalization, including embeddings, vector search, LLM-based re-ranking, and personalized explanations powered by GenAIWe offerCONTINUOUS UPSKILLING, LEARNING & DEVELOPMENTDiversity of tasks and projectsAssessment center for objective review of competency levelPersonal development planMentoring programs and leadership developmentCertification and professional development supportAccess to learning platforms including more than 2,500 internal coursesEnglish courses taught by certified teachersCORPORATE BENEFITSExtra leave daysReferral bonusesCOMPENSATION PACKAGECompetitive compensation paid in USDRegular salary and performance reviewsMEDICAL & HEALTHCAREPrivate health insuranceWell-being eventsWORKING ENVIRONMENTRecreation areas and kitchensTea, coffee and snacksSports equipment and game consolesIT EquipmentMicrosoft’s Software Assurance Home Use Program (HUP)Please note that our Talent Attraction Team reviews applications and CVs submitted in English.EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead ML/AI Solution Engineer with Databricks
Lead ML/AI Solution Engineer with Databricks

EPAM Systems • United States

Remote
USD 180,000 - 240,000
Lead ML/AI Engineer for Personalization & Databricks
Lead ML/AI Engineer for Personalization & Databricks

EPAM Systems • United States

Remote
USD 180,000 - 240,000
AI Native Solution Architect
AI Native Solution Architect

EPAM Systems Inc • Newtown (PA)

On-site
USD 150,000 - 210,000
Continuous upskilling
Personal development plan
Mentoring programs
+4
Managed Services Solution Lead / Delivery Lead
Managed Services Solution Lead / Delivery Lead

EPAM Systems Inc • Newtown (PA)

On-site
USD 150,000 - 190,000
Data Forward Deployed Engineer with Databricks
Data Forward Deployed Engineer with Databricks

EPAM Systems Inc • Newtown (PA)

On-site
USD 120,000 - 160,000
Private health insurance
Continuing education
Mentoring programs
Lead DevOps Engineer (Azure & .NET Ecosystem)
Lead DevOps Engineer (Azure & .NET Ecosystem)

EPAM Systems Inc • Newtown (PA)

On-site
USD 140,000 - 190,000
Private health insurance
Extra leave days
Referral bonuses
AI Delivery Manager
AI Delivery Manager

EPAM Systems Inc • Newtown (PA)

On-site
USD 140,000 - 190,000
Continuous upskilling
Mentoring programs
Health insurance
+1
Senior Data Technology Consultant, Databricks Data & AI Practice
Senior Data Technology Consultant, Databricks Data & AI Practice

EPAM Systems • California

Hybrid
USD 180,000 - 260,000
Extended Healthcare with Prescription 
Dental and Vision
Health Spending Account (Company Paid)
+9
Senior AI/ML Platform Engineer
Senior AI/ML Platform Engineer

Employee Relations Associates, Inc. • Denver (CO)

Hybrid
USD 170,000 - 210,000
Medical coverage
Dental coverage
Vision coverage
+4
Senior Data Technology Consultant, Databricks Data & AI Practice
Senior Data Technology Consultant, Databricks Data & AI Practice

EPAM Systems • New York (NY)

Hybrid
USD 180,000 - 220,000
Medical, Dental and Vision Insurance (
Health Savings Account
Flexible Spending Accounts
+7