Senior Data Engineer - Remote Databricks Lakehouse

INGEPSY; Capital; Presencial; COP 180.000.000 - 260.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Remote work 100%

Annual learning budget

Competitive compensation

Mentorship and tech talks

Global collaboration

Descripción de la vacante

AgileEngine is seeking a Senior Data Engineer to transform a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design batch and streaming pipelines with PySpark and Delta Lake across bronze, silver, and gold layers, leveraging AI tools to accelerate development.

You will own data quality, governance, and performance, partner with analytics teams, and help migrate legacy workloads while preserving data parity and minimizing disruption.

Formación

• 4+ years of professional experience in data engineering.
• Strong SQL and Python skills with data modeling in Lakehouse patterns.
• Experience with Databricks, Spark, Delta Lake; streaming/ETL experience.
• Familiarity with Unity Catalog and data governance practices.
• Upper-intermediate English.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark and Delta Lake.
• Model and maintain medallion architecture (bronze/silver/gold) for analytics and ML.
• Migrate legacy ETL and data warehouse workloads to Lakehouse with data parity.
• Leverage Claude or GitHub Copilot to accelerate development and testing.
• Write clean Python and SQL; ensure code quality through reviews and docs.
• Optimize Spark jobs and Delta tables for performance and cost.
• Implement data quality, governance, and lineage using Unity Catalog.
• Debug and resolve production data issues; participate in Agile delivery.

Conocimientos

Data engineering

Mentoring

English communication

Herramientas

Databricks

Apache Spark (PySpark)

Delta Lake

Python

SQL

Kafka

Airflow

Azure Data Factory

dbt

Descripción del empleo

AgileEngine is seeking a Senior Data Engineer to transform a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design batch and streaming pipelines with PySpark and Delta Lake across bronze, silver, and gold layers, leveraging AI tools to accelerate development.

You will own data quality, governance, and performance, partner with analytics teams, and help migrate legacy workloads while preserving data parity and minimizing disruption.