Lead AI Engineer with Databricks, Agentic AI

EPAM Systems, Inc.; Turkey; On-site; TRY 600,000 - 1,000,000; Full time

Benefits offered by this job

Continuous upskilling

Private health insurance

Well-being events

Referral bonuses

Job summary

EPAM Systems, Inc. seeks a Lead AI Engineer to advance agentic AI solutions within the Trinity data platform for PLM use cases.

You will embed in the client’s PLM team, design ingestion pipelines with ai_parse_document for PDFs/Excel/Word, and build Agent Bricks to extract structured data from supplier docs. You will optimize LLM-driven grouping with ai_query, develop Databricks Apps front-end (React + Python), and ensure governance through logging and lineage in Unity Catalog.

Qualifications

• 5+ years of AI/data engineering experience with Databricks (Unity Catalog, Volumes, Model Serving).
• Proficiency in ai_parse_document and Agent Bricks for document ingestion and information extraction.
• Strong Python skills (PySpark, pandas, FastAPI).
• Expertise in React front-end development with hooks and REST integration.
• Familiarity with Azure networking (Private Link/Endpoints, VNet).
• LLM prompt engineering and ai_query; Graph DBs like Neo4j; Delta Lake/Lakebase and REST API design.
• English proficiency at B2 or higher.

Responsibilities

• Embed within the client's PLM team from day one to understand workflows and properties management.
• Design and implement document ingestion pipelines using Databricks ai_parse_document for PDFs, Excel, and Word files.
• Build and maintain Agent Bricks pipelines to extract structured data.
• Develop and optimize LLM-powered grouping/recommendation logic using ai_query and Model Serving.
• Create the Databricks Apps front-end (React + Python) with upload, metadata grids, forms, and human-in-the-loop screens.
• Integrate the Graph DB layer to store relationships between specs, properties, and AI recommendations.
• Configure Azure Private Link/Private Endpoint and DNS for databricksapps.com within the corporate network.
• Implement structured logging, lineage tracking in Unity Catalog, and audit trails for governance.
• Contribute to ARB artifacts and technical design docs; collaborate with Business Analysts and Delivery Manager.

Skills

Databricks

Unity Catalog

Model Serving

Python

PySpark

React

FastAPI

REST APIs

Graph Databases

Tools

Neo4j

Azure Private Link

Unity Catalog

Job description

We are seeking a Lead AI Engineer with expertise in Databricks and Databricks Agentic AI processes to join our team. The successful candidate will play a crucial role in delivering agentic solutions within our Trinity data platform for PLM use cases. This position requires a seasoned professional with a strong background in AI engineering and a track record of leveraging Databricks for building agents, fully embedded within the PLM team to design, build and operate the core intelligence layer of the client's Product Lifecycle Management solution.ResponsibilitiesEmbed fully within the client's PLM team from day one to understand supplier specification workflows, test method structures and properties management before writing a line of codeDesign and implement the document ingestion pipeline using Databricks ai_parse_document for PDF, Excel and Word supplier specification files, storing artifacts in Unity Catalog VolumesBuild and maintain Agent Bricks (Information Extraction) pipelines to extract structured fields from supplier documents against the canonical PLM data modelDevelop and optimize LLM-powered grouping and recommendation logic using ai_query and Databricks Model Serving, to ensure outputs are explainable and auditableBuild the Databricks Apps front-end (React + Python backend), including file upload, metadata grids, dropdown-driven forms, human-in-the-loop review/approval screens and write-back to LakebaseIntegrate the Graph Database layer to store and traverse relationships between specifications, properties, test methods and AI recommendationsConfigure and maintain Azure Private Link / Private Endpoint connectivity, including DNS forwarding for databricksapps.com, to ensure all traffic stays within the client's corporate network perimeterImplement structured logging, lineage tracking in Unity Catalog, and audit trails to support Responsible AI governance requirementsContribute to ARB artefacts and technical design documentation as requiredCollaborate daily with the Business Analyst to translate PLM domain requirements into working data models and pipeline logic, and with the Delivery Manager to surface risks and effort estimatesRequirements5+ years of experience in AI/data engineering, with expertise in Databricks, including Unity Catalog, Volumes and Model ServingProficiency in ai_parse_document and Agent Bricks for document ingestion and information extractionSkills in Python, including PySpark, pandas and FastAPIExpertise in React for front-end development, including hooks and REST integrationFamiliarity with Azure networking, including Private Link, Private Endpoints and VNetCompetency in LLM prompt engineering and ai_queryKnowledge of Graph Databases such as Neo4j or equivalentBackground in Delta Lake / Lakebase and REST API design and integrationUnderstanding of Azure DevOps (ADO)English proficiency at B2 level or higherNice to haveBackground in GxP or regulated manufacturing data environmentsFamiliarity with the client's Trinity AI governance patternExperience with PLM or ERP systems (SAP, Teamcenter, Enovia)Databricks certification (Data Engineer Professional or ML Professional)Exposure to graph data modeling and lineage use casesWe offerCONTINUOUS UPSKILLING, LEARNING & DEVELOPMENTDiversity of tasks and projectsAssessment center for objective review of competency levelPersonal development planMentoring programs and leadership developmentCertification and professional development supportAccess to learning platforms including more than 2,500 internal coursesEnglish courses taught by certified teachersCORPORATE BENEFITSExtra leave daysReferral bonusesCOMPENSATION PACKAGECompetitive compensation paid in USDRegular salary and performance reviewsMEDICAL & HEALTHCAREPrivate health insuranceWell-being eventsWORKING ENVIRONMENTRecreation areas and kitchensTea, coffee and snacksSports equipment and game consolesIT EquipmentMicrosoft’s Software Assurance Home Use Program (HUP)Please note that our Talent Attraction Team reviews applications and CVs submitted in English.EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.