DataOps Engineer Specialist

ScotiaTech; Colombia; Presencial; COP 139.760.000 - 232.934.000; Jornada completa

Descripción de la vacante

Scotiatech is seeking a DataOps Engineer Specialist to join our team and modernize bank technology through robust data pipelines. You will integrate data across sources, validate end-to-end flows, and support DataOps practices to keep pipelines reliable and scalable.

You will work with SMEs to map data to Delta Lake and document lineage, while contributing to Unity Catalog cataloguing and creating basic Power BI reports to show migration progress across countries and sources.

Formación

• Bachelor’s degree in Computer Science or related field.
• Master’s degree is a plus.
• Certifications in Databricks, GCP, Azure or AWS are a plus.
• 3+ years of experience in DataOps, data integration, or data engineering support.
• Experience with data migration or cloud modernization (on-prem or GCP to Azure Databricks is a plus).
• Familiarity with CI/CD, Git, automated testing, and monitoring.

Responsabilidades

• Integrate data across sources, including legacy on‑premises to Azure Databricks, validating end‑to‑end data flow.
• Deploy and reuse ingestion, transformation, and integration patterns with architecture teams.
• Support DataOps practices (Git, CI/CD, automated testing, monitoring) for reliable pipelines.
• Explore legacy and GCP data, map source to Delta Lake, document lineage and dependencies.
• Check data quality and reconcile source vs. target across Bronze–Silver–Gold layers.
• Collaborate with SMEs to articulate data meanings and aid Unity Catalog cataloguing.
• Build simple reports/dashboards (Power BI) on data availability and migration progress by country/source/owner.

Conocimientos

DataOps

Data integration

PySpark

Python

SQL

Delta Lake

Unity Catalog

Azure Databricks

Educación

Bachelor's degree

Master's degree

Herramientas

Databricks

GCP

Azure

AWS

Terraform

Power BI

Delta Lake

Descripción del empleo

Scotiatech is a world-class engineering group! We are seeking to modernize bank technology by delivering a trustworthy, unique, and top-quality service.

We want you to be part of a Team that is always in personal and professional growth, for this reason we are looking for our new DataOps Engineer Specialist.

What will you do?

• Data Integration: Integrate data across sources—including legacy on-premises into Azure Databricks—and validate end-to-end data flow so data lands complete and usable.
• Pattern Deployment & Architecture: Deploy and reuse the team’s approved ingestion, transformation, and integration patterns, partnering with architecture and senior engineers to follow the target design and standards.
• DataOps & Automation: Support DataOps practices (Git, CI/CD, automated testing, monitoring) to keep pipelines reliable, available, and performant.
• Discovery & Mapping: Explore legacy and GCP data, produce source-to-target mappings to the Delta Lake model, and document lineage and dependencies to support migration.
• Data Quality & Reconciliation: Check data quality and reconcile source vs. target (parity), cleansing and preparing data across the medallion layers (Bronze→Silver→Gold).
• SME Collaboration & Articulation: Work with SMEs to understand and clearly articulate what data means, and support cataloguing in Unity Catalog so data is discoverable and trusted.
• Reporting & Visibility: Build simple reports/dashboards (e.g., Power BI) on data availability and migration progress—by country, source, and owner.

What do we offer you?

ducation

• Bachelor’s degree in Computer Science, Data Engineering, Information Technology, Software Engineering, or a related field.
• A Master’s degree is considered a plus.
• Relevant cloud or data engineering certifications (Databricks, GCP, Azure, AWS) are considered a plus.

Experience

• Have a B1+ level of English.
• 3+ years of experience in DataOps, data integration, or data engineering support.
• Experience integrating and mapping data, and supporting data migration or cloud modernization (on-prem or GCP to Azure Databricks is an asset).
• Familiarity with DataOps practices (Git, CI/CD, automated testing, monitoring).
• Experience working with SMEs and cross-functional teams in an Agile environment.
• Strong proficiency in Apache Spark (PySpark) and Python for large-scale data processing.
• Knowledge of data migration, cloud modernization, and platform transformation initiatives is highly desirable.
• Strong SQL skills and experience working with large-scale analytical databases and data warehouses.
• Experience with data modeling, schema design, and data warehousing concepts.
• Understanding data governance, security, monitoring, and observability practices within cloud environments.
• Experience working in Agile delivery environments and collaborating with cross-functional teams.
• Understanding of data integration, ETL/ELT, and lakehouse/medallion patterns; Azure Databricks, Delta Lake, and Unity Catalog exposure is an asset.

Preferred Qualifications

• Databricks certifications (Data Engineer Associate/Professional) are highly desirable; Azure DP-203 is an asset.
• Experience with Lakehouse architecture and Delta Lake implementation.
• Familiarity with infrastructure-as-code tools such as Terraform.
• Exposure to data quality, data observability, and monitoring frameworks.
• Experience supporting cloud migration initiatives from on-premises or legacy platforms.

If you are seeking a fulfilling opportunity to join our esteemed team and contribute to the overall success, Take the next step in your career journey and make a significant impact today!