Delta Lake Engineer - Remote Data Lakehouse & Spark Power

BairesDev; Colombia; A distancia; COP 60.000.000 - 90.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

100% remote work

Excellent USD compensation

Hardware and software setup

Flexible hours

Paid parental leaves and vacations

Diverse growth opportunities

Descripción de la vacante

BairesDev is hiring a Data Engineer to design and implement scalable data lake architectures, leveraging Delta Lake, Spark, and Databricks. The role focuses on ensuring ACID transactions, data integrity, and efficient lakehouse performance across large-scale data systems.

Ideal candidates will have 4+ years in data engineering, strong Parquet/time travel/schema-evolution knowledge, and fluent English. Enjoy remote work, flexible hours, and competitive USD compensation.

Formación

• 4+ years of experience in Software Engineering, Data Engineering, or Data Lakehouse technologies.
• Proven expertise with Delta Lake to ensure ACID transactions.
• Hands-on experience with Spark and Databricks for large-scale data processing.
• Deep understanding of Parquet formats, time travel, and schema evolution.
• Advanced proficiency in English.

Responsabilidades

• Design and implement scalable data lake architectures using Delta Lake and Apache Spark.
• Enable reliable data processing by managing ACID transactions and ensuring data consistency across the lakehouse.
• Optimize storage performance through indexing, partitioning, and vacuuming in Databricks environments.
• Develop and maintain ETL/ELT pipelines with schema evolution and time travel for historical data analysis.
• Collaborate with data teams to define storage standards and metadata management using Parquet formats.
• Ensure scalability and efficiency of large-scale data systems through performance tuning and monitoring.

Descripción del empleo

BairesDev is hiring a Data Engineer to design and implement scalable data lake architectures, leveraging Delta Lake, Spark, and Databricks. The role focuses on ensuring ACID transactions, data integrity, and efficient lakehouse performance across large-scale data systems.

Ideal candidates will have 4+ years in data engineering, strong Parquet/time travel/schema-evolution knowledge, and fluent English. Enjoy remote work, flexible hours, and competitive USD compensation.