Senior AI Engineer with Spark, AWS Services

EPAM Systems, Inc.; Turkey; On-site; TRY 300,000 - 540,000; Full time

Benefits offered by this job

Continuous upskilling

Medical insurance

English language courses

Well-being events

Job summary

EPAM Systems, Inc. is seeking a Senior AI Engineer to design and maintain data pipelines powering AI/GenAI in Risk-Based Quality Management for clinical trials.

You will own RAG ingestion, vector indexing, and data APIs, building robust ETL/ELT flows and containerized pipelines using AWS services and Spark-based analytics. The role emphasizes collaboration with data scientists and backend developers, ensuring data quality, governance, and scalable AI workflows within a production pod.

Qualifications

• Minimum 5 years of hands-on data engineering experience at scale.
• Expertise in RAG document ingestion pipelines (chunking, embedding, vector indexing).
• Proficiency in Python and Spark SQL; strong SQL skills.

Responsibilities

• Design and build RAG document ingestion pipelines (chunking, embedding, vector indexing) for clinical trial quality data.
• Build and manage vector databases (AWS OpenSearch) for RAG-powered AI workflows.
• Develop batch and streaming ETL/ELT pipelines from scratch for unstructured clinical data (PDF, DOCX).
• Build and expose data APIs for AI applications consumption.
• Optimize chunking strategies, embedding generation, and retrieval for RAG architectures.
• Manage data quality, lineage, and governance for AI/ML data pipelines.
• Deploy and maintain AWS data infrastructure (S3, Lambda, Glue, Athena, Step Functions, DynamoDB).
• Collaborate with Data Scientists and Backend Developers in an integrated pod team.

Skills

RAG pipelines

Python

Spark SQL

SQL

Docker

APIs

Cloud AWS

Tools

AWS OpenSearch

S3

Lambda

Glue

Athena

Step Functions

DynamoDB

Bedrock

API Gateway

CloudWatch

Job description

We are seeking a Senior AI Engineer with Spark and AWS Services expertise to join the RBQM Production Pod within the program. You will build and maintain data pipelines that power AI/GenAI applications for Risk-Based Quality Management in clinical trials. This role focuses on RAG document ingestion, vector indexing, and building data APIs for AI applications.ResponsibilitiesDesign and build RAG document ingestion pipelines (chunking, embedding, vector indexing) for clinical trial quality dataBuild and manage vector databases (AWS OpenSearch) for RAG-powered AI workflowsDevelop batch and streaming ETL/ELT pipelines from scratch for unstructured clinical data (PDF, DOCX, clinical reports)Build and expose data APIs for AI application consumptionOptimize chunking strategies, embedding generation, and retrieval performance for RAG architecturesManage data quality, lineage, and governance for AI/ML data pipelinesDeploy and maintain AWS data infrastructure (S3, Lambda, Glue, Athena, Step Functions, DynamoDB)Collaborate with Data Scientists and Backend Developers in an integrated pod teamRequirements5+ years of hands-on data engineering experience at scaleExpertise in RAG document ingestion pipelines (chunking, embedding, vector indexing)Proficiency in AWS OpenSearch as a vector database for RAG workflowsAdvanced proficiency in Python, including SQL and Spark SQLSkills in unstructured data transformation (PDF, DOCX) for RAG/LLM applicationsFamiliarity with AWS Services: S3, Lambda, Glue, Athena, Bedrock, Step Functions, API Gateway, CloudWatch, DynamoDBKnowledge of containerization with DockerCapability to build custom pipelines from scratch, beyond configuring out-of-the-box servicesProficiency in English at a B2+ levelNice to haveBackground in pharmaceutical or life sciences domainFamiliarity with Snowflake, Pinecone (vector DB alternative)Knowledge of SageMaker processing jobsSkills in CI/CD tools (Jenkins, Git/Bitbucket) and infrastructure tools (CDK or Terraform)Understanding of clinical data standards (CDISC, ADaM, SDTM)We offerCONTINUOUS UPSKILLING, LEARNING & DEVELOPMENTDiversity of tasks and projectsAssessment center for objective review of competency levelPersonal development planMentoring programs and leadership developmentCertification and professional development supportAccess to learning platforms including more than 2,500 internal coursesEnglish courses taught by certified teachersCORPORATE BENEFITSExtra leave daysReferral bonusesCOMPENSATION PACKAGECompetitive compensation paid in USDRegular salary and performance reviewsMEDICAL & HEALTHCAREPrivate health insuranceWell-being eventsWORKING ENVIRONMENTRecreation areas and kitchensTea, coffee and snacksSports equipment and game consolesIT EquipmentMicrosoft’s Software Assurance Home Use Program (HUP)Please note that our Talent Attraction Team reviews applications and CVs submitted in English.EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.