Senior Databricks Engineer | Hybrid Data Platform Architect

INGEPSY; Bogotá ciudad; Híbrido; COP 110.000.000 - 150.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Professional growth

Competitive compensation

A selection of exciting projects

Flextime

Descripción de la vacante

AgileEngine seeks a Senior Data Engineer to build batch and streaming pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy data workloads to a governed Lakehouse, modeling a medallion architecture powering analytics and AI use cases.

Strong SQL, Python, and Unity Catalog governance will round out the role. You will design pipelines, maintain data quality, and collaborate with DevOps and analytics engineers to ensure observability, security, and compliance while

Formación

• 3+ years of data engineering experience with Spark and cloud data architectures.
• Strong SQL and Python, data modeling for lakehouse patterns.
• Experience with streaming ingestion and orchestration tools.

Responsabilidades

• Design, build, and operate batch and streaming pipelines on Databricks.
• Model a medallion architecture serving analytics and ML.
• Migrate legacy ETL and data warehouse workloads to Lakehouse.
• Write clean code and tests; document solutions and reviews.
• Optimize Spark and Delta tables for performance and cost.
• Implement data quality, lineage, and governance controls.

Conocimientos

Apache Spark

SQL

Python

Databricks

Delta Lake

Delight in data modeling

Structured Streaming

Kafka

Data governance

Code quality

Herramientas

Databricks

Airflow

Azure Data Factory

dbt

Unity Catalog

Terraform

Descripción del empleo

AgileEngine seeks a Senior Data Engineer to build batch and streaming pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy data workloads to a governed Lakehouse, modeling a medallion architecture powering analytics and AI use cases.

Strong SQL, Python, and Unity Catalog governance will round out the role. You will design pipelines, maintain data quality, and collaborate with DevOps and analytics engineers to ensure observability, security, and compliance while