Senior Data Analyst

TechDoQuest; Toronto; On-site; CAD 100,000 - 140,000; Full time

Job summary

TechDoQuest is seeking a Senior Data Analyst to lead a large-scale migration from GCP BigQuery to Azure Databricks. You will own data discovery, source-to-target mapping, data lineage, migration planning, reconciliation, and business validation across enterprise data domains.

You will collaborate with Architects and Engineering teams to validate migration outcomes, support UAT, and produce governance artifacts such as data dictionaries and lineage models.

Qualifications

• 12+ years of data analysis experience.
• Proficiency with GCP BigQuery and SQL.
• Experience with Apache Airflow / Cloud Composer.
• Spark / PySpark expertise.
• Data profiling and data mapping capabilities.

Responsibilities

• Lead data analysis, profiling, discovery, and migration assessment activities.
• Perform source-to-target mapping for BigQuery datasets, pipelines, and reporting assets.
• Analyze Airflow/Composer workflows and downstream dependencies.
• Define migration, reconciliation, and data quality requirements.
• Collaborate with Business SMEs, Architects, and Engineering teams to validate migration outcomes.
• Support User Acceptance Testing (UAT), defect analysis, data validation, and migration sign-off activities.
• Create data lineage, business rules, data dictionaries, and governance documentation.

Skills

GCP BigQuery

Apache Airflow / Cloud Composer

Spark / PySpark

SQL & Data Analysis

Data Profiling & Data Mapping

Job description

We are seeking a Senior Data Analyst to play a pivotal role in a large-scale enterprise migration from GCP BigQuery to Azure Databricks. This position will focus on data discovery, source-to-target mapping, data lineage analysis, migration planning, reconciliation, and business validation activities across various enterprise data domains. The successful candidate will be instrumental in migrating BigQuery datasets, Airflow/Composer workloads, and analytics assets to an Azure Databricks Lakehouse platform.

Responsibilities

• Lead data analysis, profiling, discovery, and migration assessment activities.
• Perform source-to-target mapping for BigQuery datasets, pipelines, and reporting assets.
• Analyze Airflow/Composer workflows and downstream dependencies.
• Define migration, reconciliation, and data quality requirements.
• Collaborate with Business SMEs, Architects, and Engineering teams to validate migration outcomes.
• Support User Acceptance Testing (UAT), defect analysis, data validation, and migration sign-off activities.
• Create data lineage, business rules, data dictionaries, and governance documentation.

Mandatory Skills

• GCP BigQuery
• Apache Airflow / Cloud Composer
• Spark / PySpark
• SQL & Data Analysis
• Data Profiling & Data Mapping

Preferred Skills

• Experience with Enterprise Data Lake / Lakehouse architectures
• Familiarity with data governance frameworks
• Knowledge of data visualization tools

Qualifications

• 12+ years of Data Analysis experience