Lead Data Engineer

goremutualinsurance; Cambridge; Vor Ort; CAD 120.000 - 160.000; Vollzeit

Zusammenfassung

Gore Mutual is seeking a senior Data Engineer to design, develop, and maintain scalable data infrastructure on the Databricks lakehouse. You will lead data pipelines across policy, billing, claims and finance to support reporting, pricing and analytics.

You will collaborate with data scientists, product owners, and business stakeholders to deliver high-quality, secure, and observable data solutions, while mentoring engineers and driving best practices in TDD, CI/CD and data governance.

Qualifikationen

• Bachelor’s or Master’s degree in CS/Data/SWE or related field.
• Demonstrated ownership of data initiatives end to end and mentoring experience.

Aufgaben

• Design, develop and maintain scalable data infrastructure on the Databricks lakehouse.
• Build scalable data pipelines spanning policy, billing, claims and finance data for reporting, pricing and analytics.
• Implement data quality checks within pipelines to ensure data integrity and accuracy.
• Collaborate with data scientists, data product owners, and stakeholders to understand data requirements and deliver high-quality data solutions.
• Automate data management processes and address data security.
• Optimize data pipelines for performance and reliability.
• Ensure health, monitoring and observability of data platforms and apply security best practices.

Kenntnisse

Python

Spark

SQL

Data modeling

Data pipelines

CI/CD

TDD

Data quality

Observability

Ausbildung

Bachelor's or Master's degree in Computer Science, Data Engineering, Software Engineering or related field

Tools

Azure Databricks

Unity Catalog

Delta Lake

Lakeflow

dbt

Airflow

Azure DevOps

Azure Storage

Key Vault

Jobbeschreibung

At Gore Mutual, we've always set ourselves apart as a modern mutual that does good. Now, we're proudly building on that legacy to transform our company-and our industry-for the better.

Effective January 1, 2026, Gore has joined Beneva-the country's largest mutual insurance company-as part of its Property & Casualty operations in Ontario and Western Canada. During 2026, Gore will combine its operations with Unica Insurance, Beneva's Ontario-based subsidiary specializing in niche commercial and personal insurance, creating a stronger, more diversified mutual insurer with greater scale and long-term stability.

Every decision and investment remains anchored in long-term benefits to customers, members, and communities. Come join us.

This position requires a strong technical background and is responsible for designing, developing and maintaining scalable data infrastructure on the Databricks lakehouse , the enterprise single source of truth. It plays a pivotal role in enabling data-driven decision making across the enterprise and in providing technical leadership to the data engineering team, with a focus on delivering business value. This role requires close collaboration with cross-functional stakeholders, IT and data science teams.

What will you do in this role:

Data Modelling, Infrastructure and Design

Develop and maintain scalable, reliable, and resilient systems to evolve and mature our data infrastructure and tech stack.

Develop configuration driven, scalable, efficient, and robust data pipelines spanning policy, billing, claims and finance data to support reporting, pricing and analytics.

Develop and implement data quality checks within pipelines to ensure the integrity and accuracy of incoming data.

Data Strategy

Collaborate with cross-functional teams, including data scientists, data product owners, and business stakeholders to understand their data requirements, prioritize work by business value, and deliver high-quality data pipeline solutions, including build-versus-buy recommendations where appropriate .

Automating Data Management Processes and Handling Data Security

Automate data management processes and handle data security.

Ensure that security protocols and best practices are in place to protect against potential security threats.

Testing and Optimizing Data Platform

Optimize data pipelines to ensure efficient data flow.

Ensure that the data extracted from sources is accurate , complete, and usable. This might involve checking for missing values, inconsistent formats, or anomalies that could indicate errors.

Test the efficiency and speed of data pipelines and databases. This can help identify bottlenecks and optimize performance.

Verify that different components of the data infrastructure work together as expected. This includes checking that data flows correctly from sources to databases, and from databases to applications.

Maintenance of Data Platform

Regularly check the health and performance of the data platform. This can involve tracking metrics like query times, error rates, and resource usage.

Identify and resolve issues that arise in the data platform. This can involve debugging code, optimizing queries, or adjusting configurations.

Keep the data platform and engineering practices up to date with the latest technologies and security patches, including AI-assisted development (coding agents, context engineering and agent skills) where it improves delivery and quality.

Implement strategies to protect data from loss or corruption. This can involve regular backups, redundancy measures, and disaster recovery plan

What you need to succeed in this role:

• Bachelor's or Master's degree in Computer Science , Data Engineering, Software Engineering or a related field.
• A minimum of 5-8 years of relevant experience as a data engineer is required . This includes experience in data engineering, data system development, or related roles, with demonstrated ownership of initiatives end to end and experience mentoring other engineers.
• Strong understanding of data structures, data modeling, and software architecture, with sound judgment on when to build and when to buy.
• Deep, hands-on knowledge of Azure Databricks (Unity Catalog, Delta Lake, Lakeflow /Jobs, compute and cost optimization) and related Microsoft Azure services (DevOps, Azure Storage, Key Vault, etc.)
• Strong problem-solving and critical-thinking abilities, with a high sense of ownership and a focus on delivering business value and return on investment.
• Experience with software design patterns, test-driven development (TDD), CI/ CD and operating data pipelines as production systems (data quality, observability, incident response).
• Proficiency in Python, including a strong grasp of Object Oriented and Functional programming paradigms.
• A solid understanding of Spark concepts and distributed systems, including data transformations, DataFrames , and Spark SQL; experience with dbt and Airflow is an asset.
• Excellent SQL