Senior Databricks Data Engineer - PySpark & GCP Expert

Agileengine; Capital; Presencial; COP 364.275.000 - 496.738.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Professional growth

Competitive compensation

Fortune 500 projects

Flextime

Descripción de la vacante

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to design scalable ETL/ELT pipelines and operationalize agentic workflows for data automation.

You will integrate with existing Databricks, GCP, BigQuery, and Delta Lake environments; develop data processing solutions, and build reusable data engineering components for multiple use cases. Strong English and production-grade delivery are required.

Formación

• 5+ years hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• Hands-on GCP experience, esp. BigQuery.
• Delta Lake and lakehouse architectures knowledge.
• ETL/ELT design, performance optimization, data quality.
• Experience delivering production-grade data solutions.
• Strong analytical and troubleshooting abilities.
• Software engineering practices: testing, version control, deployment, monitoring.

Responsabilidades

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering processes.
• Integrate Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines to support new business requirements and datasets.
• Create reusable data engineering frameworks across use cases.
• Implement data quality checks, monitoring, and production controls.
• Optimize PySpark and SQL workloads for performance and scalability.
• Support testing, deployment, and productionizing of data solutions.
• Troubleshoot complex data and production issues; implement sustainable fixes.
• Collaborate with business, data engineering, and platform teams to find more automation opportunities.

Conocimientos

Databricks

PySpark

SQL

BigQuery

Delta Lake

ETL/ELT design

Data quality

Problem solving

CI/CD & monitoring

English

Descripción del empleo

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to design scalable ETL/ELT pipelines and operationalize agentic workflows for data automation.

You will integrate with existing Databricks, GCP, BigQuery, and Delta Lake environments; develop data processing solutions, and build reusable data engineering components for multiple use cases. Strong English and production-grade delivery are required.