Data Engineer - Databricks Lakehouse & Pipelines

AgileEngine, LLC.; Perímetro Urbano Barranquilla; Presencial; COP 289.119.000 - 417.617.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Remote work with flexible hours

Modern projects with leading brands

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine, LLC. is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse.

You will design batch and streaming pipelines with PySpark and Delta Lake, leveraging a medallion architecture for analytics and ML consumers, and migrate legacy workloads with minimal disruption. You’ll use Claude and GitHub Copilot to accelerate development, write clean Python/SQL, and collaborate with DevOps and analytics teams to ensure governance, quality, and

Formación

• 3+ years of professional data engineering experience with Spark-based architectures.
• Strong SQL and Python skills with data modeling experience.
• Experience migrating legacy pipelines to modern lakehouse platforms and data governance.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks with PySpark and Delta Lake.
• Maintain medallion architecture (bronze/silver/gold) for analytics and ML workloads.
• Migrate legacy ETL workloads to Lakehouse with validated data parity and minimal disruption.
• Collaborate with DevOps and analytics teams on observability, security, and compliance.

Conocimientos

Data engineering

Problem solving

Communication

Herramientas

Databricks

Apache Spark (PySpark)

Delta Lake

SQL

Python

Kafka

Databricks Workflows

Airflow

dbt

Descripción del empleo

AgileEngine, LLC. is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse.

You will design batch and streaming pipelines with PySpark and Delta Lake, leveraging a medallion architecture for analytics and ML consumers, and migrate legacy workloads with minimal disruption. You’ll use Claude and GitHub Copilot to accelerate development, write clean Python/SQL, and collaborate with DevOps and analytics teams to ensure governance, quality, and