AI-Powered Data Engineer for Enterprise Discovery

Bristol Myers Squibb; Princeton (NJ); On-site; USD 88,000 - 106,000; Full time

Benefits offered by this job

Comprehensive benefits

Flexibility

Growth opportunities

Job summary

Bristol Myers Squibb in Princeton, NJ is hiring an AI / Data Engineer to join the Data Discovery Services team within Enterprise Data Platforms. You will build pipelines, enrich metadata with taxonomy, and enable retrieval-augmented generation with vector embeddings, contributing to AI-powered discovery across the enterprise.

This hands-on role focuses on Python development, data pipelines, search optimization, and API development to expose metadata capabilities for AI agents and apps.

Qualifications

• Bachelor’s degree in computer science, Data Science, Engineering, or related field; Master’s preferred.
• Proven data engineering or software engineering experience delivering production pipelines.
• Proficient Python; primary language.
• Proficient SQL.
• Experience with Databricks and AWS Glue.
• Strong data engineering fundamentals: ETL/ELT, data modeling, data quality, orchestration.
• Familiarity with AWS services (S3, Lambda, API Gateway, Glue).
• Experience with OpenSearch or Elasticsearch.
• Understanding metadata management and data cataloging.
• Strong problem-solving and willingness to learn.
• Good communication and teamwork.

Responsibilities

• Build Python pipelines that pull metadata from enterprise data catalogs, enrich with taxonomy tags, and publish to the discovery platform.
• Tune and optimize search indexes to ensure results match user needs.
• Build a semantic knowledge layer with vector embeddings to grow a data inventory supporting retrieval-augmented generation (RAG).
• Maintain integrations that sync ontology and taxonomy changes into the discovery platform.
• Investigate and resolve data pipeline issues across Databricks and AWS Glue, add data quality checks.
• Build API endpoints and MCP servers that expose search and metadata capabilities.
• Design metadata pipelines that map cross-domain dataset relationships and add to the cross-domain join catalog with confidence scores.
• Analyze search patterns and improve the discovery experience so the system learns and improves over time.

Skills

Python

SQL

Data engineering

ETL/ELT

OpenSearch/Elasticsearch

Databricks

AWS Glue

Communication

Education

Bachelor’s degree in computer science, Data Science, Engineering, or related field

Master’s degree preferred

Tools

Databricks

AWS Glue

OpenSearch

S3

Lambda

API Gateway

Docker

Job description

Bristol Myers Squibb in Princeton, NJ is hiring an AI / Data Engineer to join the Data Discovery Services team within Enterprise Data Platforms. You will build pipelines, enrich metadata with taxonomy, and enable retrieval-augmented generation with vector embeddings, contributing to AI-powered discovery across the enterprise.

This hands-on role focuses on Python development, data pipelines, search optimization, and API development to expose metadata capabilities for AI agents and apps.