Data Engineer: Databricks, GCP & AI Agentic Workflows

AgileEngine, LLC.; Porto; Hybrid; EUR 50,000 - 70,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to design, develop, and productionize scalable ETL/ELT pipelines and agentic workflows.

You will work across Databricks, GCP, BigQuery, and Delta Lake, building reusable data engineering components and ensuring data quality. The role emphasizes scalable data solutions, performance optimization, and collaboration with cross-functional teams to automate data workflows and support evolving business needs.

Qualifications

• 5+ years of hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• Hands-on GCP experience, especially BigQuery.
• Experience with Delta Lake and data lake architectures.
• Strong ETL/ELT, pipeline design, performance optimization, and data quality.
• Production-grade data solutions development.
• Strong analytical and troubleshooting abilities.
• Understanding of software engineering practices (testing, VCS, deployment, monitoring).

Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines.
• Build and operationalize agentic workflows to automate data processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake.
• Develop pipelines and solutions for new business requirements.
• Build reusable frameworks across multiple data use cases.
• Implement data quality checks, monitoring, validation, and production controls.
• Optimize PySpark and SQL workloads for performance and scalability.
• Support testing, deployment, productionization and enhancements.
• Troubleshoot complex data and production issues.
• Collaborate with business and platform teams to identify more automation.

Skills

Databricks

PySpark

Advanced SQL

BigQuery

Delta Lake

ETL/ELT

Data pipelines

Data quality

Testing & deployment

Version control

Production support

Troubleshooting

Automation

Job description

AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to design, develop, and productionize scalable ETL/ELT pipelines and agentic workflows.

You will work across Databricks, GCP, BigQuery, and Delta Lake, building reusable data engineering components and ensuring data quality. The role emphasizes scalable data solutions, performance optimization, and collaboration with cross-functional teams to automate data workflows and support evolving business needs.