Data Engineer

Blanc Labs; Bogotá ciudad; Presencial; COP 90.000.000 - 140.000.000; Jornada completa

Descripción de la vacante

Blanc Labs is seeking a Data Engineer to design, build, and maintain scalable data pipelines on Azure Databricks and Synapse. You will implement CDC for real-time data synchronization and write Python code to process data, transform it, and load it into Delta Lake and ADLS Gen2.

You will collaborate with architects and analysts to translate business needs into robust solutions, monitor performance, and ensure data quality across ingestion, transformation, and storage layers.

Formación

• 3+ years of experience in data engineering or a related field.
• Hands-on with Azure Databricks for data processing and analytics workloads.
• Proficiency with Apache Spark for large‑scale distributed data processing.
• Experience implementing CDC solutions (e.g., Debezium, Azure SQL CDC, or Delta Live Tables).
• Strong programming skills in Python for data engineering tasks.
• Solid understanding of SQL and relational databases (Azure SQL).
• Experience building ETL/ELT pipelines using Azure Data Factory or Databricks Workflows.
• Knowledge of data modeling and data warehousing concepts (Azure Synapse).
• Strong analytical and problem‑solving skills.
• Good communication across tech and business teams.

Responsabilidades

• Design, build, and maintain scalable data pipelines using Azure Databricks and Apache Spark.
• Develop and optimize ETL/ELT processes to ingest, transform, and load data from diverse sources.
• Implement CDC solutions for real-time data synchronization using Debezium, Azure SQL CDC, or Delta Live Tables.
• Write clean, efficient Python code for data processing and automation.
• Collaborate with data architects and analysts to translate requirements into robust pipelines on Azure Synapse/Databricks.
• Monitor and optimize pipeline performance, reliability, and cost using Azure Monitor and Databricks observability.
• Ensure data quality and integrity across ingestion, transformation, and storage (ADLS Gen2, Delta Lake).
• Support integration of structured/unstructured data into Microsoft Fabric / Azure Synapse platforms.
• Participate in code reviews and promote best practices in data engineering.
• Document pipelines, transformations, and architecture for team visibility.

Conocimientos

Azure Databricks

Apache Spark

Python

SQL

CDC implementation

Azure Data Factory

Azure Synapse Analytics

Data modeling

Herramientas

Debezium

Delta Live Tables

Databricks Workflows

Azure Data Lake Storage

Descripción del empleo

Blanc Labs is a premier partner for global enterprises, leading the way in digitization, automation, and the development of next-generation digital products and services. Our expertise in digital transformation powers businesses to accelerate service delivery, drive customer engagement, and foster growth.

We are seeking a skilled Data Engineer with expertise in modern data platforms and real-time data integration to design, build, and maintain scalable data pipelines that power enterprise analytics and business intelligence. This role is ideal for someone who enjoys solving complex data engineering challenges and thrives working with distributed data processing frameworks.

What You’ll Be Doing

• Design, build, and maintain scalable data pipelines using Azure Databricks and Apache Spark
• Develop and optimize ETL/ELT processes to ingest, transform, and load data from diverse source systems using Azure Data Factory and Azure Databricks
• Implement Change Data Capture (CDC) solutions to enable real-time and near-real-time data synchronization, leveraging tools such as Azure SQL CDC, Debezium, or Databricks native CDC (Delta Live Tables)
• Write clean, efficient, and maintainable code in Python for data processing and automation
• Collaborate with data architects and analysts to translate business requirements into robust data pipeline designs on Azure Synapse Analytics / Azure Databricks
• Monitor, troubleshoot, and optimize data pipeline performance, reliability, and cost efficiency using Azure Monitor and Databricks observability tools
• Ensure data quality, consistency, and integrity across ingestion, transformation, and storage layers, including Azure Data Lake Storage (ADLS Gen2) and Delta Lake
• Support integration of structured and unstructured data sources into unified data platforms built on Microsoft Fabric / Azure Synapse
• Participate in code reviews and promote data engineering best practices across the team
• Document data pipelines, transformations, and architecture for team visibility and maintainability
• Collaborate with cross-functional teams to support analytics, reporting (Power BI), and downstream AI/ML use cases

What You’ll Need To Be Successful

• 3+ years of experience in data engineering or a related field
• Strong hands-on experience with Azure Databricks for data processing and analytics workloads
• Proficiency with Apache Spark for large-scale distributed data processing
• Experience implementing CDC solutions and tools (e.g., Debezium, Azure SQL CDC, or Delta Live Tables)
• Strong programming skills in Python for data engineering tasks
• Solid understanding of SQL and relational database concepts, particularly within Azure SQL Database / SQL Managed Instance
• Experience building and maintaining ETL/ELT pipelines in production environments using Azure Data Factory and/or Databricks Workflows
• Understanding of data modeling principles and data warehousing concepts, including Azure Synapse Analytics dimensional/lakehouse modeling
• Strong analytical and problem-solving skills
• Good communication skills, with the ability to work across technical and business teams

Blanc Labs is enabling a digital future. Headquartered in Toronto, we partner with clients in North & South America to digitize and automate their operations and build their next generation of digital products and services. We empower clients to enhance their digital offerings and bring creative solutions to the market faster. Learn more at www.blanclabs.com.

Blanc Labs is an equal opportunity employer and is committed to providing employment in accordance with the Ontario Human Rights Code and the Accessibility for Ontarians with Disabilities Act. Accommodations within reason due to a disability or medical need are available on request for candidates taking part in the recruitment process.