Data Engineer - Databricks & GCP Automation

AgileEngine, LLC.; Sur; Híbrido; COP 404.013.000 - 606.020.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Professional growth

Competitive compensation

A selection of exciting projects

Flextime

Descripción de la vacante

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation. You will design, build, and optimize pipelines, integrate with BigQuery and Delta Lake, and implement robust data quality and production controls.

The role emphasizes creating reusable data frameworks, improving performance, and collaborating across business and platform teams to drive automation initiatives.

Formación

• 5+ years hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• GCP familiarity, especially BigQuery.
• Experience with Delta Lake and data lake architectures.
• Strong analytical and troubleshooting capabilities.
• Understanding of testing, deployment, monitoring, and production support.

Responsabilidades

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake.
• Develop data pipelines and processing solutions for new business requirements.
• Create reusable frameworks and components for cross-use cases.
• Implement data quality checks, monitoring, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, productionization, and ongoing enhancements.

Conocimientos

Databricks

PySpark

SQL

Data quality

Troubleshooting

Software engineering practices

Herramientas

BigQuery

Delta Lake

GCP

Databricks environment

Descripción del empleo

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation. You will design, build, and optimize pipelines, integrate with BigQuery and Delta Lake, and implement robust data quality and production controls.

The role emphasizes creating reusable data frameworks, improving performance, and collaborating across business and platform teams to drive automation initiatives.