Data Engineering Lead

Builders; Atlanta (GA); Presencial; USD 140 000 - 180 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Competitive Salary

Bonus Structure

Medical, Dental, Vision Insurance

401K with Company Match

PTO

Resumo da oferta

Builders is seeking an experienced Data Platform Lead to own and advance our Azure Databricks lakehouse for property and casualty data. You will architect bronze/silver/gold pipelines, drive modernization, governance alignment, and lead engineering teams to deliver scalable, secure data foundations.

You will orchestrate large data migrations from Guidewire and legacy systems, model P&C entities, and enable actuarial and finance use cases with robust data quality and lineage.

Qualificações

• Associate degree or an equivalent combination of education and experience.
• 10+ years of data engineering experience, including 6+ years hands-on with Databricks (PySpark, Spark SQL, Delta Lake).
• Databricks Data Engineer Professional certification; Azure data or solutions architect certification preferred.
• Strong knowledge of Unity Catalog: governance model, permissions, row/column-level security, lineage, and system tables.
• Experience leading large data migrations from core insurance and legacy systems into a lakehouse or data warehouse, with P&C domain knowledge.

Responsabilidades

• Design the Databricks lakehouse architecture using bronze/silver/gold layers on Delta Lake for policy, rating, claims, billing, reinsurance, and customer data.
• Plan and lead phased migration from Guidewire and legacy systems into Azure Databricks, including historical loads, incremental/CDC loads, reconciliation, and decommissioning support.
• Model core property and casualty insurance entities: accounts, policies, insured risks and exposures, coverages, endorsements, premiums, claims, reserves, payments, recoveries.
• Provide data for Actuarial and Finance use cases including written/earned/unearned premium, loss triangles, IBNR, loss ratios, Schedule P and NAIC reporting.
• Integrate third-party sources such as geospatial and weather data.
• Define and operationalize Unity Catalog design: metastore, catalogs, schemas, environments, naming conventions, permissions, security, and lineage.
• Manage pipeline orchestration and operations in Databricks, including Databricks Workflows, scheduling, dependencies, retries, alerting and SLA monitoring, and runbooks.
• Own performance and cost work: Spark tuning, partitioning, liquid clustering, Photon, and cluster/SQL warehouse sizing.
• Partner with IT and Platform Engineer on storage, networking, security, identity, IaC, and CI/CD.
• Set standards for pipelines, testing, code review, and documentation; lead mentoring for Data Engineers and contractors.
• Collaborate with data governance to implement access controls, PII masking, lineage, and data quality rules.

Conhecimentos

Databricks

PySpark

Spark SQL

Delta Lake

SQL

Python

Data Modeling

Leadership

Formação académica

Associate degree

Ferramentas

Unity Catalog

Databricks Workflows

Terraform

Azure Data Factory

Git

Descrição da oferta de emprego

Builders is committed to reliability, fairness, and customer-first service, earning an A rating by AM Best and recognition through Top Workplaces “Culture Excellence” honors. This role supports an insurance business built on strong partnerships and mutual respect, with an onsite opportunity in Atlanta, GA to lead a platform that helps teams modernize and govern data for property and casualty insurance.

As the technical owner of the Azure Databricks lakehouse platform, you will drive architecture, modernization, governance alignment, and engineering leadership to deliver a scalable, secure, reliable, high-performing foundation for property and casualty data.

What you’ll do

• Design the Databricks lakehouse architecture using a bronze/silver/gold approach on Delta Lake for policy, rating, claims, billing, reinsurance, and customer data.
• Plan and lead phased migration from Guidewire (PolicyCenter, ClaimCenter, BillingCenter) and legacy systems into Azure Databricks, including historical loads, incremental/CDC loads, reconciliation, and decommissioning support.
• Model core property and casualty insurance entities: accounts, policies, insured risks and exposures, coverages, endorsements, premiums, claims, reserves, payments, and recoveries.
• Provide data for Actuarial and Finance use cases including written/earned/unearned premium, loss triangles, IBNR (incurred but not reported) inputs, loss ratios, and Schedule P and NAIC statutory reporting.
• Integrate third-party sources such as geospatial and weather data.
• Define and operationalize Unity Catalog design: metastore, catalogs, schemas, development/test/production environments, naming conventions, permissions, row/column-level security, and lineage.
• Manage pipeline orchestration and operations in Databricks, including Databricks Workflows (Lakeflow Jobs), scheduling, dependencies, retries, alerting and SLA monitoring, and production failure handling with runbooks.
• Own performance and cost work: Spark tuning, partitioning, liquid clustering, Photon, and cluster/SQL warehouse sizing.
• Partner with IT and a Platform Engineer on infrastructure for storage, networking, security, identity, IaC, and CI/CD.
• Set standards for pipelines, testing, code review, and documentation, and lead mentoring and review for Data Engineers and junior contractors.
• Collaborate with the data governance lead to implement access controls, PII masking, lineage, and data quality rules.
• Perform other duties as assigned.

What you bring

• Associate degree or an equivalent combination of education and experience.
• 10+ years of data engineering experience, including 6+ years hands-on with Databricks (PySpark, Spark SQL, Delta Lake), plus experience leading or mentoring engineering teams.
• Databricks Data Engineer Professional certification; Azure data or solutions architect certification is preferred.
• Strong knowledge of Unity Catalog: governance model, permissions, row/column-level security, lineage, and system tables.
• Experience leading large data migrations from core insurance and legacy systems into a lakehouse or data warehouse, with property and casualty domain knowledge and familiarity with the policy and claims lifecycle and core P&C KPIs.
• Knowledge of data modeling including dimensional modeling, data vault, SCD patterns, and insurance data models.
• Expert technical skills across Spark internals and performance tuning, streaming (structured streaming, Auto Loader), CDC patterns, MERGE, and Lakeflow/Delta Live Tables.
• Expert-level SQL and Python, with software engineering practices such as testing, code review, and modular design.
• Knowledge of data infrastructure: ADLS Gen2, Azure Data Factory, Event Hubs, Key Vault, Entra ID, VNets, and Private Link.
• Experience scheduling and monitoring production pipelines with Databricks Workflows and/or Azure Data Factory, including dependencies, parameterization, and monitoring with Databricks system tables.
• Experience with Git-based version control (GitHub or Bitbucket), including branching strategies, pull request reviews, and Databricks Git folders.
• Infrastructure-as-code and CI/CD for data using Terraform, Databricks Asset Bundles, Azure DevOps, GitHub Actions, or Bitbucket Pipelines.
• Experience with Guidewire data (Guidewire DataHub/InfoCenter, Cloud Data Access) is preferred, including supporting actuarial reserving and pricing; API and integration tools such as MuleSoft are preferred.
• Knowledge of insurance data standards such as ACORD is preferred.

Benefits at Builders

• Competitive Salary
• Bonus Structure
• Profit Sharing
• Medical, Dental, Vision Insurance
• Employer Paid Short Term Disability
• Employer Paid Long Term Disability
• Employer Paid Life Insurance
• Voluntary Life Insurance
• 401K with Company Match
• PTO

Equal Opportunity

Builders Insurance Group is an equal opportunity employer. Applicants are welcomed from all walks of life, and the company does not discriminate based on any protected status. This is part of building a diverse and inclusive workplace.