DataOps Engineer Specialist- ScotiaTech

Scotiabank; Bogotá; Híbrido; COP 70.000.000 - 120.000.000; Jornada completa

Descripción de la vacante

Scotiabank seeks a DataOps Engineer to enable data integration and DataOps for International Banking, driving reliable data flow into Azure Databricks. You will partner with SMEs and senior engineers to map legacy data for migration with confidence.

You will work across Data Integration, Pattern Deployment, DataOps automation, and governance, delivering clear source-to-target mappings and dashboards to show migration progress by country and owner.

Formación

• 3+ years of DataOps, data integration, or data engineering experience.
• Experience mapping data and supporting data migration to cloud.
• Strong SQL and large-scale data warehousing knowledge.
• Familiarity with Agile environments.
• Experience with cross-functional collaboration.

Responsabilidades

• Data Integration across sources and validation of end-to-end data flow.
• Deploy ingestion/transformation patterns with architecture guidance.
• Support DataOps practices (Git, CI/CD, automated testing, monitoring).
• Discover and map legacy and GCP data for Delta Lake migration.
• Ensure data quality and parity across Bronze-Silver-Gold.
• Collaborate with SMEs to articulate data meaning and catalog via Unity Catalog.
• Build dashboards showing data availability and migration progress by country and owner.

Conocimientos

Azure Databricks

PySpark

Python

SQL

Data modeling

Delta Lake

Unity Catalog

Git CI/CD

Cloud data migration

Educación

Bachelor's degree in Computer Science

Master's degree

Herramientas

Databricks

Azure

GCP

AWS

Descripción del empleo

Purpose

The DataOps Engineer Specialistenables data integration and DataOps for International Banking, supporting the reliable flow of data across systems and into Harbour (Azure Databricks). Working hands-on with architecture and senior engineers, this role integrates data, deploys approved data patterns, and helps discover, map, and articulate legacy on-premises and GCP data so it can be migrated with confidence. Reporting to the Senior Data Engineer, the incumbent partners with SMEs and the engineering team to make data trusted, well-understood, and ready for use across International Banking markets.

Accountabilities

Data Integration: Integrate data across sources—including legacy on-premisesinto Azure Databricks—and validate end-to-end data flow so data lands complete and usable.

Pattern Deployment & Architecture: Deploy and reuse the team’s approved ingestion, transformation, and integration patterns, partnering with architecture and senior engineers to follow the target design and standards.

DataOps & Automation: Support DataOps practices (Git, CI/CD, automated testing, monitoring) to keep pipelines reliable, available, and performant.

Discovery & Mapping: Explore legacy and GCP data, produce source-to-target mappings to the Delta Lake model, and document lineage and dependencies to support migration.

Data Quality & Reconciliation: Check data quality and reconcile source vs. target (parity), cleansing and preparing data across the medallion layers ( Bronze→Silver→Gold ).

SME Collaboration & Articulation: Work with SMEs to understand and clearly articulate what data means, and support cataloguing in Unity Catalog so data is discoverable and trusted.

Reporting & Visibility: Build simple reports/dashboards (e.g., Power BI) on data availability and migration progress—by country, source, and owner.

Dimensions

Colombia, Canada

Business Unit: IBTT

Education / Experience / Other Information

Education

Bachelor’s degree in Computer Science, Data Engineering, Information Technology, Software Engineering, or a related field.

A Master’s degree is considered a plus.

Relevant cloud or data engineering certifications (Databricks, GCP, Azure, AWS) are considered a plus.

Experience

3+ years of experience in DataOps, data integration, or data engineering support.

Experience integrating and mapping data, and supporting data migration or cloud modernization (on-prem or GCP to Azure Databricks is an asset).

Familiarity with DataOps practices (Git, CI/CD, automated testing, monitoring).

Experience working with SMEs and cross-functional teams in an Agile environment.

Strong proficiency in Apache Spark (PySpark) and Python for large-scale data processing.

Knowledge of data migration, cloud modernization, and platform transformation initiatives is highly desirable.

Strong SQL skills and experience working with large-scale analytical databases and data warehouses.

Experience with data modeling, schema design, and data warehousing concepts.

Understanding data governance, security, monitoring, and observability practices within cloud environments.

Experience working in Agile delivery environments and collaborating with cross-functional teams.

Understanding of data integration, ETL/ELT, and lakehouse/medallion patterns; Azure Databricks, Delta Lake, and Unity Catalog exposure is an asset.

Preferred Qualifications

Databricks certifications (Data Engineer Associate/Professional) are highly desirable; Azure DP-203 is an asset.

Experience with Lakehouse architecture and Delta Lake implementation.

Familiarity with infrastructure-as-code tools such as Terraform.

Exposure to data quality, data observability, and monitoring frameworks.

Experience supporting cloud migration initiatives from on-premises or legacy platforms.

Soft Skills

Strong stakeholder management and communication skills.

Excellent analytical thinking and problem-solving abilities.

Adaptability and willingness to learn new technologies in a rapidly evolving data ecosystem.

Ability to work effectively in collaborative, distributed, and multicultural teams.

Proactive mindset with a focus on continuous improvement and operational excellence.

Strong ownership and accountability for deliverables and outcomes.