Senior Databricks Lakehouse Engineer (Azure)

Pacific Northwest National Laboratory; Columbia (SC); Hybrid; USD 133,000 - 210,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k) with company match

Relocation Assistance

Paid Holidays

Vacation Leave (120 hours/yr)

Tuition Assistance

Job summary

Pacific Northwest National Laboratory is seeking a Databricks Engineer to design, build, and operate a scalable lakehouse. You’ll enable analytics with ERP data, Power BI, and AI/ML, focusing on governed, reliable data products and modern MPP workflows.

You will modernize legacy warehouses, shape CI/CD for Databricks, and configure workspaces, clusters, and jobs while embedding data quality and observability for mission critical pipelines.

Qualifications

• At least 1 year of experience developing production Databricks solutions.
• Experience with Python and SQL in a Spark/Databricks environment.
• Experience with cloud services (AWS, Azure, GCP).

Responsibilities

• Design, build, and operate Databricks lakehouse solutions for analytics.
• Develop scalable data pipelines from source to gold tables.
• Implement governance, security, and data quality measures.

Skills

Databricks

Python

SQL

Spark/PySpark

Delta Lake

Unity Catalog

Azure Cloud

CI/CD (DAB, GitHub Actions)

Education

PhD in Data/Software Engineering

MS/MA in Data/Software Engineering

BS/BA in Data/Software Engineering

AA in Data/Software Engineering

HS/GED + 16 years experience

Tools

Databricks

Azure

Delta Live Tables

Unity Catalog

Job description

Pacific Northwest National Laboratory is seeking a Databricks Engineer to design, build, and operate a scalable lakehouse. You’ll enable analytics with ERP data, Power BI, and AI/ML, focusing on governed, reliable data products and modern MPP workflows.

You will modernize legacy warehouses, shape CI/CD for Databricks, and configure workspaces, clusters, and jobs while embedding data quality and observability for mission critical pipelines.