Senior Data Engineer: Databricks, GCP & AI Workflows (Remote)

AgileEngine, LLC.; Madrid; Hybrid; EUR 65,000 - 98,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to build scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.

Ideal candidates have 5+ years with Databricks, PySpark, SQL proficiency, and hands-on BigQuery experience within modern lakehouse architectures. Remote-friendly and client-focused projects await.

Qualifications

• 5+ years of hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• Hands-on GCP experience, especially BigQuery.
• Experience with Delta Lake and data lakehouse architectures.
• Strong ETL/ELT, data pipeline design, performance optimization and data quality.
• Production-grade data solutions experience.
• Solid analytical and troubleshooting skills.
• Understanding of software engineering practices: testing, VCS, deployment, monitoring, and production support.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operational processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines and processing solutions for new business requirements and datasets.
• Create reusable frameworks and components for multiple data use-cases.
• Implement data quality checks, monitoring, validation, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, productionization, and ongoing enhancements.
• Troubleshoot complex data and production issues and implement sustainable solutions.
• Collaborate with business, data engineering, and platform teams to identify automation opportunities.

Skills

Databricks

PySpark

SQL

BigQuery

Delta Lake

ETL/ELT

Data pipelines

Tools

BigQuery

Job description

AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to build scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.

Ideal candidates have 5+ years with Databricks, PySpark, SQL proficiency, and hands-on BigQuery experience within modern lakehouse architectures. Remote-friendly and client-focused projects await.