Principal Databricks Lakehouse Architect - AI & GenAI

Berkshire Hathaway Specialty Insurance Company; Boston (MA); Presencial; USD 200.000 - 240.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Comprehensive Health, Dental andVision

Disability Insurance

Life Insurance

AD&D Insurance

401(k) with Company Match

Paid Time Off

Descripción de la vacante

Berkshire Hathaway Specialty Insurance (BHSI) is modernizing its data ecosystem, pursuing a Databricks-based Lakehouse to enable insight-driven decisions and AI-enabled outcomes at scale. We seek a Principal Data Architect to own the Lakehouse architecture, secure and optimize workloads, and drive Generative AI readiness.

This senior IC role focuses on technical leadership across data engineering, analytics, and AI/ML, reporting to the Global Head of Data Analytics Solutions.

Formación

• 15+ years in data engineering, data architecture, or software architecture.
• 5+ years architecting on the Databricks Lakehouse Platform.
• Deep expertise in Spark, Delta Lake, Unity Catalog, MLflow.

Responsabilidades

• Define and evolve the enterprise Lakehouse reference architecture on Databricks across data engineering, analytics, and AI workloads.
• Lead design and rollout of Databricks AI capabilities including vector search and model serving.
• Implement governance, access controls and auditing with Unity Catalog.
• Establish CI/CD and infrastructure-as-code standards for data pipelines and ML models.
• Set architecture standards and mentor engineers and data scientists.

Conocimientos

Python

SQL

Data architecture

Cloud platforms (AWS/Azure)

Databricks Lakehouse

Herramientas

Databricks Lakehouse Platform

Delta Lake

Unity Catalog

MLflow

Terraform

Apache Spark

Descripción del empleo

Berkshire Hathaway Specialty Insurance (BHSI) is modernizing its data ecosystem, pursuing a Databricks-based Lakehouse to enable insight-driven decisions and AI-enabled outcomes at scale. We seek a Principal Data Architect to own the Lakehouse architecture, secure and optimize workloads, and drive Generative AI readiness.

This senior IC role focuses on technical leadership across data engineering, analytics, and AI/ML, reporting to the Global Head of Data Analytics Solutions.