Data Engineer — Databricks, PySpark & GCP Specialist

Agileengine; Perímetro Urbano Barranquilla; Hybrid; COP 281,233,000 - 397,035,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Projects with Fortune 500 clients

Flextime

Job summary

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to design scalable ETL/ELT pipelines and productionize agentic workflows for data automation.

You will work across Databricks, BigQuery, Delta Lake, and SQL to build robust pipelines, implement data quality checks, and optimize workloads for performance and scale. Mature English communication and collaboration are essential.

Qualifications

• 5+ years hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• Hands-on with GCP, especially BigQuery.
• Experience with Delta Lake and modern lakehouse architectures.
• Strong ETL/ELT design and data quality understanding.
• Production-grade data solutions experience.
• Strong analytical and troubleshooting abilities.
• Knowledge of software engineering practices: testing, version control, deployment, monitoring.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering tasks.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines to support new datasets and business requirements.
• Create reusable frameworks and components for multiple use cases.
• Implement data quality checks, monitoring, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, and productionization of data solutions.
• Troubleshoot complex data issues and implement sustainable fixes.
• Collaborate with business, data engineering, and platform teams to identify automation opportunities.

Skills

Analytical thinking

Troubleshooting

Communication

Team collaboration

English proficiency

Tools

Databricks

PySpark

SQL

GCP

BigQuery

Delta Lake

ETL/ELT design

Job description

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to design scalable ETL/ELT pipelines and productionize agentic workflows for data automation.

You will work across Databricks, BigQuery, Delta Lake, and SQL to build robust pipelines, implement data quality checks, and optimize workloads for performance and scale. Mature English communication and collaboration are essential.