Remote Senior Databricks Data Engineer: Lakehouse & AI

AgileEngine; Colombia; Presencial; COP 120.000.000 - 180.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth opportunities

Competitive compensation

Remote flexibility

Modern projects

Collaborative culture

Well-being support

Descripción de la vacante

AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

This role also uses AI tools like Claude and GitHub Copilot to speed up development, and requires strong collaboration with cross-functional teams to ensure data quality and governance.

Formación

• Strong data modeling skills across dimensional and Lakehouse patterns.
• Mentor junior engineers and explain data concepts to non-technical stakeholders.
• Experience with Databricks, Spark (PySpark), and Delta Lake in cloud environments.
• Upper-intermediate English is required for collaboration across teams.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
• Model and maintain a medallion (bronze/silver/gold) architecture serving analytics and ML.
• Migrate legacy ETL and data warehouse workloads onto the Lakehouse with minimal disruption.
• Write clean, well-tested Python and SQL; participate in code reviews and documentation.

Conocimientos

Data modeling

Mentoring juniors

Problem solving

English (upper‑intermediate)

Herramientas

Databricks

Apache Spark (PySpark)

Delta Lake

Databricks Workflows

Airflow

Azure Data Factory

Unity Catalog

Kafka / Event Hubs

Descripción del empleo

AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

This role also uses AI tools like Claude and GitHub Copilot to speed up development, and requires strong collaboration with cross-functional teams to ensure data quality and governance.