Data Engineer - Databricks, GCP & Automation Pipelines

AgileEngine, LLC.; Cartagena de Indias; Presencial; COP 202.007.000 - 303.010.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Professional growth

Competitive compensation

A selection of exciting projects

Flextime

Descripción de la vacante

AgileEngine is seeking a Data Engineer to design and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL. You will build agentic workflows and integrate with GCP, BigQuery, Delta Lake, and your data environments.

This role offers exposure to Fortune 500 projects, opportunities for growth, flexible hours, and a people-focused culture. You will collaborate with data and platform teams to automate operations and improve data quality in production.

Formación

• 5+ years of hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• GCP experience, especially BigQuery.
• Delta Lake and modern data lake architectures.
• Strong ETL/ELT, data pipeline design, and data quality understanding.
• Proven ability to build scalable, production-grade data solutions.
• Strong analytical and troubleshooting skills.
• Knowledge of testing, version control, deployment, monitoring, and production support.

Responsabilidades

• Design, develop, and optimize scalable ETL/ELT data pipelines with Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operations.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake.
• Develop data pipelines to support new business requirements and datasets.
• Create reusable frameworks and components for cross-use across data projects.
• Implement data quality checks, monitoring, validation, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, productionization, and ongoing enhancements.
• Troubleshoot complex data and production issues and implement durable solutions.
• Collaborate with business, data engineering, and platform teams to identify more automation.

Conocimientos

Databricks & PySpark

Advanced SQL

GCP BigQuery

Delta Lake

ETL/ELT design

Production-grade data pipelines

Troubleshooting

DevOps / CI-CD

Herramientas

Databricks

BigQuery

Delta Lake

GCP

Descripción del empleo

AgileEngine is seeking a Data Engineer to design and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL. You will build agentic workflows and integrate with GCP, BigQuery, Delta Lake, and your data environments.

This role offers exposure to Fortune 500 projects, opportunities for growth, flexible hours, and a people-focused culture. You will collaborate with data and platform teams to automate operations and improve data quality in production.