Senior Data Engineer — Real-Time Data Platform

Deel; Colombia; Presencial; COP 404.013.000 - 606.020.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Stock grant opportunities

Flexible office membership (IWG)

Descripción de la vacante

Deel is building a world-class data platform to turn operational data into fast, reliable data products. You will own streaming jobs, manage compute clusters, and keep lakehouse tables healthy while driving cost efficiency.

You will design end-to-end streaming workflows, ensure data correctness, and collaborate with partner teams to deliver scalable analytics capabilities across the business.

Formación

• 7+ years of data engineering experience, including production streaming
• Strong Python and SQL skills; Scala or Java a plus
• Experience with CDC and Kafka semantics

Responsabilidades

• Design, build, and harden Spark Structured Streaming applications processing CDC events in near real-time
• Run and scale AWS EMR clusters and tune resources to optimize cost
• Own health of Apache Iceberg tables, data partitioning, compaction and cleanup
• Onboard new domains to the streaming platform and verify backfills in Snowflake
• Debug production issues, identify root causes, and implement durable fixes
• Build observability and tooling to shorten time to diagnosis
• Improve deployment and developer UX via CI/CD and IaC
• Handle ad hoc ingestion and backfill requests from partner teams

Conocimientos

Spark Structured Streaming

AWS EMR

Iceberg

Python

SQL

Kafka

CI/CD

Airflow

Scala/Java

debugging

Herramientas

Estuary Flow

Snowflake

Grafana Loki

Descripción del empleo

Deel is building a world-class data platform to turn operational data into fast, reliable data products. You will own streaming jobs, manage compute clusters, and keep lakehouse tables healthy while driving cost efficiency.

You will design end-to-end streaming workflows, ensure data correctness, and collaborate with partner teams to deliver scalable analytics capabilities across the business.