Data Engineer Specialist

Scotiabank; Bogotá ciudad; Presencial; COP 90.000.000 - 140.000.000; Jornada completa

Descripción de la vacante

ScotiaTech, el centro tecnológico de Scotiabank en Bogotá, busca un Especialista en Data Engineering para diseñar, construir y mantener pipelines de datos en Azure Databricks y Delta Lake, asegurando calidad, rendimiento y cumplimiento. Trabajarás con equipos multifuncionales para entregar soluciones escalables y seguras en una arquitectura lakehouse.

Se valoran 3–5 años de experiencia en ingeniería de datos en la nube, dominio de PySpark y SQL, así como habilidades en modelado de datos y

Formación

• 3–5 años de experiencia en Data Engineering en entornos cloud.
• Experiencia práctica con Azure Databricks, Delta Lake y Spark.
• Sólidos conocimientos de PySpark y Python para procesamiento de datos a gran escala.
• Conocimientos de modelado de datos, esquemas y warehousing.

Responsabilidades

• Diseñar, desarrollar y mantener pipelines ETL/ELT en Azure y Azure Databricks para apoyar productos de datos.
• Colaborar con Data Scientists, Product Owners y equipos de Arquitectura para entregar soluciones de datos de alta calidad.
• Modelar datos, esquemas y transformaciones (Data Vault y dimensional) para analytics.
• Monitorear y optimizar pipelines, incluir tuning de desempeño y FinOps de Spark.
• Asegurar gobernanza, seguridad y cumplimiento en entornos Azure (Unity Catalog, RBAC, cifrado).
• Resolver incidencias en pipelines y realizar análisis de causa raíz.
• Promover DevOps/DataOps, documentación y testing automatizado.
• Apoyar migración de cargas a Azure Databricks minimizando riesgos.

Conocimientos

Azure Databricks

Python (PySpark)

Apache Spark

SQL

Data modeling

Datalake/Data warehouse

Unity Catalog

Lakehouse

DevOps/DataOps

Git & CI/CD

Educación

Bachelor en Ciencias de la Computación/Data Engineering/IT

Maestría deseable

Certificaciones en Databricks/Azure/AWS/GCP

Herramientas

Azure Databricks

Delta Lake

Unity Catalog

Spark

Python

Descripción del empleo

ID de la solicitud: 271754

Gracias por tu interés en ScotiaTech, el nuevo e innovador centro tecnológico de Scotiabank en Bogotá.

Únete a un equipo ganador que promueve la creatividad y la innovación en un entorno cambiante, donde siempre estamos comprometidos con el logro de resultados, en una cultura inclusiva, diversa y de alto desempeño.

Purpose

• The Data Engineer Specialist designs, develops, and maintains data pipelines and infrastructure on Microsoft Azure and Azure Databricks to support the creation of scalable data products for International Banking. Reporting to the Senior Data Engineer, this role focuses on implementing robust, efficient, and secure lakehouse-based data solutions that enable data-driven decision-making and support business objectives across International Banking markets, while contributing to the migration and rationalization of legacy workloads onto the Azure Databricks target platform.

Accountabilities

• Build, test, and maintain data pipelines and ETL/ELT processes on Azure and Azure Databricks (Delta Lake, medallion architecture) to ensure reliable and efficient data flow for data products.
• Collaborate with the Senior Data Engineer and cross-functional teams (Data Scientists, Product Owners, Architecture, and Platform Engineering) to understand requirements and deliver high-quality data solutions aligned with international business goals.
• Implement data models, schemas, and transformations (dimensional and Data Vault patterns) to support analytics and reporting needs.
• Ensure data quality, integrity, and performance by monitoring and optimizing data pipelines, including performance tuning and cost optimization (FinOps) of Spark workloads.
• Adhere to and help apply data governance, security, and compliance standards within Azure environments, including Unity Catalog, role-based access control, data lineage, encryption, and key management.
• Troubleshoot and resolve issues in data pipelines to minimize downtime and ensure operational efficiency, performing root-cause analysis where needed.
• Contribute to the adoption of best practices and tools for data engineering, including documentation, automated testing, and DevOps/DataOps practices.
• Support the migration and rationalization of legacy workloads (including existing Google Cloud Platform estates) onto the Azure Databricks target platform, minimizing business and regulatory risk.
• Stay updated on modern Azure and Databricks data engineering trends and tools to enhance pipeline capabilities.

Reporting Relationships

Primary Manager

Direct Reports

NA

Shared Reports

NA

Dimensions

NA

Education / Experience / Other Information (include only those that are specific to the role)

Education

• Bachelor’s degree in Computer Science, Data Engineering, Information Technology, Software Engineering, or a related field.
• A Master’s degree is considered a plus.
• Relevant cloud or data engineering certifications (Databricks, GCP, Azure, AWS) are considered a plus.

Experience

• 3-5 years of experience in Data Engineering, designing and developing scalable data solutions in cloud environments.
• Hands-on experience with Azure Databricks, including data pipeline development, Delta Lake, Spark, notebooks, workflows, Unity Catalog, and Lakehouse architecture.
• Strong proficiency in Apache Spark (PySpark) and Python for large-scale data processing.
• Experience building batch and streaming data pipelines using modern data engineering practices.
• Knowledge of data migration, cloud modernization, and platform transformation initiatives is highly desirable.
• Strong SQL skills and experience working with large-scale analytical databases and data warehouses.
• Experience with data modeling, schema design, and data warehousing concepts.
• Familiarity with DevOps and DataOps practices, including Git-based version control, CI/CD pipelines, automated testing, and deployment processes.

Competencies

Working Conditions

Work in a standard office-based environment; non-standard hours are a common occurrence

Ubicación(s): Colombia : Bogota : Bogota ScotiaTech es una unidad de negocio de ScotiaGBS, un grupo de empresas de Scotiabank, ubicado en Bogotá, Colombia. ScotiaTech fue creado para apoyar diversos sistemas y procesos tecnológicos del Banco. Ofrecemos un entorno de trabajo inclusivo y positivo, además de ventajas competitivas. En ScotiaTech, valoramos las habilidades y experiencias únicas que cada persona aporta y nos hemos comprometido a crear y mantener un entorno inclusivo y accesible para todos. Agradecemos a todos los candidatos por su interés en esta oportunidad profesional en ScotiaTech; sin embargo, solo contactaremos a quienes hayan sido seleccionados para una entrevista.