Senior Data Engineer Lead Scotiatech

GSG-GESTIÓN DE SERVICIOS GLOBALES S.A.S; Bogotá; Presencial; COP 90.000.000 - 170.000.000; Jornada completa

Descripción de la vacante

GSG-GESTIÓN DE SERVICIOS GLOBALES S.A.S. seeks a Senior Data Engineer to design, build, and maintain scalable data products and pipelines on Microsoft Azure and Azure Databricks.

You will lead a team, establish best practices, and ensure secure, compliant lakehouse solutions that empower data-driven decisions across markets. You will own end-to-end solution design, mentor engineers, and collaborate with Product Owners and Architects to deliver high-quality data platforms while optimizing cost,

Formación

• 5+ years in Data Engineering with 3+ on Azure/Azure Databricks.
• Proven track record leading technical projects.
• Strong experience with Azure Databricks, Spark, Delta Lake, Unity Catalog, Workflows.
• Hands-on Python, PySpark, SQL; data modeling and lakehouse architectures.
• Experience with CI/CD, Git, Terraform or Bicep; IaC principles.
• Knowledge of cloud security, RBAC, data privacy, and compliance.
• Experience with cloud migration projects is highly valued.
• Excellent English communication (B2+).

Responsabilidades

• Design, build, and optimize data pipelines and lakehouse solutions on Azure Databricks.
• Lead technical design and delivery from concept to production.
• Mentor Data Engineers through coaching and code reviews.
• Collaborate with Product Owners, Data Scientists, Architects, and Eng teams.
• Implement data governance, security, and access management practices.
• Monitor platforms for reliability, performance, data quality, and cost.
• Drive DataOps, DevOps, and CI/CD adoption for cloud-native solutions.
• Troubleshoot complex issues and implement long-term fixes.
• Support cloud migration of legacy/GCP workloads to Azure Databricks.
• Evaluate new technologies to strengthen data capabilities.

Conocimientos

Leadership

Stakeholder management

Communication

Ownership

Problem solving

Mentoring

Educación

Bachelor's degree in Computer Science/IT/Data Science

Herramientas

Azure Databricks

Delta Lake

Unity Catalog

Workflows

Azure Data Factory

Azure Data Lake Storage Gen2

Event Hubs

Azure Synapse

Terraform or Bicep

Git

Descripción del empleo

The Senior Data Engineer is responsible for designing, building, and maintaining scalable data products and pipelines using Microsoft Azure and Azure Databricks, supporting key International Banking initiatives. This role combines deep technical expertise with team leadership, driving the delivery of secure, high-quality lakehouse solutions that enable data-driven decision-making across multiple markets.

As the technical lead, you will own end-to-end solution design and implementation, establish engineering best practices, mentor a team of Data Engineers, and ensure compliance with enterprise architecture, security, and regulatory standards.

Key Responsibilities

• Design, develop, and optimize data pipelines and lakehouse solutions using Azure Databricks, Delta Lake, and Medallion Architecture.
• Lead the technical design and delivery of data engineering initiatives from concept to production.
• Mentor and support Data Engineers through coaching, code reviews, and technical guidance.
• Collaborate with Product Owners, Data Scientists, Architects, and Engineering teams to build scalable data products.
• Implement and maintain best practices for data governance, security, compliance, and access management.
• Monitor data platforms to ensure reliability, performance, data quality, and cost efficiency.
• Drive adoption of modern data engineering practices, including ETL/ELT, DataOps, DevOps, CI/CD, and cloud-native solutions.
• Troubleshoot complex data platform issues and implement long-term solutions.
• Support cloud migration initiatives, including the transition of legacy and GCP-based workloads to Azure Databricks.
• Evaluate emerging technologies and recommend improvements to strengthen data capabilities.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field.
• 5+ years of hands-on experience in Data Engineering, with at least 3 years working with Azure and/or Azure Databricks.
• Proven experience leading technical projects and mentoring engineering teams.
• Strong expertise in Azure Databricks, Spark, Delta Lake, Unity Catalog, Workflows, Azure Data Factory, Azure Data Lake Storage Gen2, Event Hubs, and Azure Synapse.
• Advanced proficiency in Python, PySpark, and SQL.
• Experience with data modeling, data warehousing, and lakehouse architectures.
• Knowledge of CI/CD pipelines, Git, Infrastructure as Code (Terraform or Bicep), and automated testing.
• Understanding of cloud security, governance, RBAC, data privacy, and compliance frameworks.
• Experience with cloud migration projects is highly valued.
• Strong English communication skills (B2+ or higher).

Preferred Qualifications

• Azure or Databricks certifications (e.g., DP-203, Databricks Certified Data Engineer Professional).
• Experience in financial services or banking environments.
• Spanish language skills are considered an asset.

Soft Skills

• Strong leadership and stakeholder management capabilities.
• Excellent communication and collaboration skills.
• Results-driven mindset with a strong sense of ownership and accountability.
• Ability to manage multiple priorities in fast-paced environments.
• Proactive problem solver with a continuous improvement mindset.
• Comfortable working with distributed and multicultural teams.