Senior Data Engineer — Databricks on Azure Lakehouse

PROSTAFF Schweiz GmbH; United States; Remote; USD 168,000 - 252,000; Full time

Job summary

PROSTAFF Schweiz GmbH is seeking a Senior Data Engineer to build a modern data platform for insurance and reinsurance, leveraging Databricks on Azure to centralize data and enable reliable reporting and AI use cases.

You will design scalable lakehouse architectures, manage Bronze-Silver-Gold layers, ensure data quality and governance, and collaborate with analytics, reporting and business teams.

Qualifications

• Several years of experience as a Senior Data Engineer or in a comparable role.
• Proficient in designing and building scalable data platforms.

Responsibilities

• Build, maintain and optimize data pipelines in Databricks.
• Design and implement scalable Lakehouse and Medallion Architecture concepts.
• Structure data into meaningful business domains.
• Develop and maintain Bronze, Silver and Gold data layers.
• Create Gold layers to support applications, reports and analytics.
• Integrate data from SAP sources.
• Implement data quality checks, monitoring and validation mechanisms.
• Support data modelling, storage concepts and performance optimization.
• Contribute to governance, security and access management concepts.
• Support CI/CD pipelines for data engineering workflows.
• Assist reporting requirements, likely with Power BI and/or Databricks.
• Prepare the data platform for future AI and advanced analytics use cases.
• Enable onboarding of business users onto the data platform.
• Collaborate with infrastructure, architecture, business, reporting and data teams.
• Act as a senior, independent, structured professional with strong communication skills.

Skills

Senior experience

Data pipelines

Stakeholder comms

Data governance

Data modeling

Tools

Databricks

Power BI

Azure

CI/CD in data engineering

Job description

PROSTAFF Schweiz GmbH is seeking a Senior Data Engineer to build a modern data platform for insurance and reinsurance, leveraging Databricks on Azure to centralize data and enable reliable reporting and AI use cases.

You will design scalable lakehouse architectures, manage Bronze-Silver-Gold layers, ensure data quality and governance, and collaborate with analytics, reporting and business teams.