Data Engineer Specialist

ScotiaTech; Colombia; Presencial; COP 90.000.000 - 140.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth opportunities

Diverse environment

Benefits for well-being

Competitive wages

Descripción de la vacante

Scotiatech in Colombia is seeking a Data Engineer Specialist to design, build, and maintain scalable data pipelines on Azure Databricks, enabling reliable data flow for analytics and reporting.

You will collaborate with data scientists, product owners, and platform teams to implement data models, ensure data quality, security, and governance, and drive DevOps/DataOps practices across cloud environments.

Formación

• 3-5 years of experience in Data Engineering
• Hands-on experience with Azure Databricks and Spark
• Strong SQL and data warehousing knowledge
• Experience with data governance and security practices

Responsabilidades

• Build, test, and maintain data pipelines on Azure Databricks.
• Collaborate with data scientists and product teams to deliver data solutions.
• Implement models, schemas, and transformations for analytics.
• Monitor pipelines for quality, performance, and cost efficiency.
• Enforce data governance and security in Azure environments (Unity Catalog, RBAC).
• Troubleshoot issues and perform root-cause analyses.
• Advise on DevOps/DataOps practices and automation.

Conocimientos

Data pipelines

Azure Databricks

PySpark

Python

SQL

Agile

DevOps

Herramientas

Delta Lake

Unity Catalog

Git

CI/CD

Descripción del empleo

Scotiatech is a world-class engineering group! We are seeking to modernize bank technology by delivering a trustworthy, unique, and top-quality service.

We want you to be part of a Team that is always in personal and professional growth, for this reason we are looking for our new Data Engineer Specialist.

What will you do?

• Build, test, and maintain data pipelines and ETL/ELT processes on Azure and Azure Databricks (Delta Lake, medallion architecture) to ensure reliable and efficient data flow for data products.
• Collaborate with the Senior Data Engineer and cross-functional teams (Data Scientists, Product Owners, Architecture, and Platform Engineering) to understand requirements and deliver high-quality data solutions aligned with international business goals.
• Implement data models, schemas, and transformations (dimensional and Data Vault patterns) to support analytics and reporting needs.
• Ensure data quality, integrity, and performance by monitoring and optimizing data pipelines, including performance tuning and cost optimization (FinOps) of Spark workloads.
• Adhere to and help apply data governance, security, and compliance standards within Azure environments, including Unity Catalog, role-based access control, data lineage, encryption, and key management.
• Troubleshoot and resolve issues in data pipelines to minimize downtime and ensure operational efficiency, performing root-cause analysis where needed.
• Contribute to the adoption of best practices and tools for data engineering, including documentation, automated testing, and DevOps/DataOps practices.
• Support the migration and rationalization of legacy workloads (including existing Google Cloud Platform estates) onto the Azure Databricks target platform, minimizing business and regulatory risk.
• Stay updated on modern Azure and Databricks data engineering trends and tools to enhance pipeline capabilities.

What do we offer you?

• Growth opportunities
• Diverse, dynamic and multicultural environment
• Benefits for your financial and emotional well-being
• competitive wages

What do we expect from you?

Experience

• Have a B1 level of English.
• 3-5 years of experience in Data Engineering, designing and developing scalable data solutions in cloud environments.
• Hands-on experience with Azure Databricks, including data pipeline development, Delta Lake, Spark, notebooks, workflows, Unity Catalog, and Lakehouse architecture.
• Strong proficiency in Apache Spark (PySpark) and Python for large-scale data processing.
• Experience building batch and streaming data pipelines using modern data engineering practices.
• Knowledge of data migration, cloud modernization, and platform transformation initiatives is highly desirable.
• Strong SQL skills and experience working with large-scale analytical databases and data warehouses.
• Experience with data modeling, schema design, and data warehousing concepts.
• Familiarity with DevOps and DataOps practices, including Git-based version control, CI/CD pipelines, automated testing, and deployment processes.
• Understanding data governance, security, monitoring, and observability practices within cloud environments.
• Experience working in Agile delivery environments and collaborating with cross-functional teams.
• Strong problem-solving skills and ability to troubleshoot complex data processing challenges.