AI Data Engineer: Build AI-Ready Pipelines & Governance

Howard Hughes Medical Institute; Kentucky; Hybride; USD 120 000 - 180 000; Plein temps

Résumé du poste

Howard Hughes Medical Institute (HHMI) seeks an experienced AI Data Engineer to build and operate the data platforms powering AI across HHMI's knowledge and data fabric. You will implement pipelines, medallion architecture, and governance for AI-facing data assets, working under the Principal Knowledge & Data Architect.

This hybrid role requires hands-on production data engineering, Databricks/Spark expertise, and collaboration with the AI Developer and Operations teams.

Qualifications

• Bachelor’s degree or equivalent; 4+ years of hands-on data engineering experience.
• Production pipelines owned and operated in real environments.
• Experience with AI/data foundation work is ideal.
• Strong knowledge of Databricks platforms and cloud data tooling.

Responsabilités

• Build the AI-facing data pipelines: ingestion, transformation, and governance-ready data.
• Implement the medallion architecture with Delta Lake and scheduled materializations.
• Own the workflow orchestration framework using Databricks Workflows and Delta Live Tables.
• Implement governance patterns including Unity Catalog and access controls.
• Build and operate retrieval-supporting infrastructure like embeddings and vector stores.
• Partner with Operations Capabilities on source-system contracts and data interfaces.
• Design and operate data quality, freshness, drift monitoring, and alerting.
• Support AI Developer velocity by enabling data access for product teams.
• Contribute to and evolve reusable reference patterns and templates.

Connaissances

Production data engineering

Databricks & Spark depth

Python

SQL

IaC with Terraform

AWS fundamentals

Formation

Bachelor’s degree or equivalent

Outils

Databricks

Spark

Delta Lake

Delta Live Tables

Unity Catalog

Workflows

Airflow

Git

CI/CD

Terraform

AWS S3

KMS

Description du poste

Howard Hughes Medical Institute (HHMI) seeks an experienced AI Data Engineer to build and operate the data platforms powering AI across HHMI's knowledge and data fabric. You will implement pipelines, medallion architecture, and governance for AI-facing data assets, working under the Principal Knowledge & Data Architect.

This hybrid role requires hands-on production data engineering, Databricks/Spark expertise, and collaboration with the AI Developer and Operations teams.