Senior Databricks Engineer: Lakehouse & AI Pipelines

Pacific Northwest National Laboratory; United States; Sur place; USD 120 000 - 180 000; Plein temps

Avantages offerts par ce poste

Medical insurance

Dental insurance

Vision insurance

Résumé du poste

Pacific Northwest National Laboratory seeks an experienced Databricks Engineer to design, build, and operate a lakehouse powering analytics and reporting across PNNL systems. The role focuses on governed, reliable data products and enabling downstream analytics with Power BI and AI/ML.

You will modernize data warehouses, transform ETL pipelines, and shape CI/CD for Databricks, while ensuring security, data quality, and observability across environments. A collaborative WE Culture is emphasized.

Qualifications

• At least 1 year of experience developing production Databricks solutions.
• Experience in coding languages, such as Python and SQL in a Spark/Databricks environment.
• Experience with cloud services (platforms like AWS, Azure, GCP).

Responsabilités

• Design, build, and operate enterprise data lakehouse powering analytics and reporting.
• Deliver governed, reliable, and performant data products for ERP and other enterprise sources.
• Enable downstream analytics with Power BI and AI/ML.
• Migrate legacy data warehouses and ETL tools to Databricks.
• Configure and operate Databricks workspaces, clusters, jobs, and workflows; tune for performance and reliability.
• Embed data quality, monitoring, and observability in pipelines.
• Implement role-based access controls, data masking, and Unity Catalog models for secure data sharing and governance.
• Stay current with Azure and Databricks capabilities; pilot Delta Live Tables and Unity Catalog enhancements.
• Collaborate in WE Culture with agile, mentoring environment.

Connaissances

Lakehouse Architecture & Data Eng

Databricks & Azure Platform Eng

Data Modernization & Integration

Governance, Security & Reliability

AI & Automation

Formation

PhD or higher

MS/MA

BS/BA

AA

HS/GED

Outils

Delta Lake

Unity Catalog

Databricks

GitHub Actions

CI/CD

Python

SQL

Spark/PySpark

Azure

DAB (Databricks Asset Bundles)

Description du poste

Pacific Northwest National Laboratory seeks an experienced Databricks Engineer to design, build, and operate a lakehouse powering analytics and reporting across PNNL systems. The role focuses on governed, reliable data products and enabling downstream analytics with Power BI and AI/ML.

You will modernize data warehouses, transform ETL pipelines, and shape CI/CD for Databricks, while ensuring security, data quality, and observability across environments. A collaborative WE Culture is emphasized.