Senior Data Architect – Databricks / Azure / Data Integration

ThoughtStorm Inc.; Edmonton; On-site; CAD 140,000 - 190,000; Full time

Job summary

ThoughtStorm Inc. seeks a Senior Data Architect to lead data architecture and integration for the Government of Alberta’s Digital Regulatory Assurance System (DRAS).

You will design the target-state architecture linking DRAS, Databricks, and the Alberta Geospatial Platform. The role requires hands-on work with Databricks, Delta Lake, Spark/PySpark, SQL, and Unity Catalog, plus developing comprehensive data models and enterprise integrations using REST APIs, ETL/ELT, and CDC.

Qualifications

• 8+ years of enterprise Data Architecture/database experience.
• Strong communication and leadership skills.
• Experience with large, complex government or public-sector environments.
• Hands-on experience with Databricks, Delta Lake, Spark/PySpark, SQL, and Unity Catalog.
• Proven ability to design target-state architectures connecting multiple platforms (DRAS, Databricks, Alberta Geospatial Platform).

Responsibilities

• Lead data architecture and integration for DRAS initiatives.
• Design target-state data architecture connecting DRAS with Databricks and Alberta Geospatial Platform.
• Develop enterprise data models (conceptual to lakehouse).
• Design enterprise integration using REST APIs, ETL/ELT, CDC, and event streaming.
• Implement data governance, metadata, lineage, security, and RBAC.
• Collaborate with public-sector teams and manage stakeholder expectations.

Skills

Databricks

Delta Lake

Spark/PySpark

SQL

Unity Catalog

REST APIs

ETL/ELT

Change Data Capture

Data governance

leadership

Tools

ADLS

Azure Data Factory

Event Hubs

Key Vault

Job description

• Seeking a Senior Data Architect to lead data architecture and integration for the Government of Alberta’s Digital Regulatory Assurance System (DRAS).
• Design target-state data architecture connecting DRAS, Databricks Data Management Platform, and Alberta Geospatial Platform.
• Strong hands-on expertise with Databricks, Delta Lake, Spark/PySpark, SQL, and Unity Catalog is required.
• Develop conceptual, logical, physical, relational, dimensional, and lakehouse data models for enterprise data environments.
• Design enterprise integration using REST APIs, ETL/ELT, event streaming, batch/near-real-time processing, and Change Data Capture (CDC).
• Strong Microsoft Azure experience with technologies such as ADLS, Azure Data Factory, Event Hubs, and Key Vault is highly preferred.
• Experience with geospatial data/platforms, spatial data flows, geometry, coordinate reference systems, and spatial standards is important.
• Apply data governance, metadata, lineage, data quality, master/reference data, security, privacy, and role-based access controls.
• Experience with ServiceNow Integration, enterprise architecture, SQL databases/data warehouses, Agile delivery, and public-sector/government environments is highly desirable.
• Candidate should have 8+ years of enterprise Data Architecture/database experience, strong communication/leadership skills, and experience working with large, complex organizations.