Remote Data Engineer: Databricks Lakehouse & Pipelines

INGEPSY; Perímetro Urbano Barranquilla; Presencial; COP 60.000.000 - 120.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits: mentorship,Tech

Competitive compensation with reviews

100% remote with flexible hours

Modern technologies with global teams

Collaborative culture with recognition

Well-being & support tailored to your

Descripción de la vacante

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, supporting analytics, reporting, and ML consumers.

You will work with Claude and GitHub Copilot to accelerate development, implement data quality and governance, and collaborate with DevOps and analytics engineers on observability, security, and compliance.

Formación

• 3+ years of professional experience in data engineering with Spark and cloud architectures.
• Strong hands-on experience building data pipelines with Databricks, Spark (PySpark) and Delta Lake.
• Advanced SQL and Python with data modeling skills across dimensional and Lakehouse patterns.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks.
• Model and maintain a medallion architecture for analytics, reporting, and ML consumers.
• Migrate legacy ETL workloads onto the Lakehouse with validated data parity.

Conocimientos

Databricks

PySpark

Delta Lake

SQL

Python

Structured Streaming

Airflow

Azure Data Factory

Kafka

Unity Catalog

dbt

Security best practices

Herramientas

Terraform

PostgreSQL

Azure DevOps

Descripción del empleo

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, supporting analytics, reporting, and ML consumers.

You will work with Claude and GitHub Copilot to accelerate development, implement data quality and governance, and collaborate with DevOps and analytics engineers on observability, security, and compliance.