Data Engineer - Databricks & GCP Automation (Remote/Flex)

AgileEngine, LLC.; Barcelona; Hybrid; EUR 85,000 - 120,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine is seeking a Data Engineer to design and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL. You will build agentic workflows to automate data engineering tasks and integrate Databricks, GCP, BigQuery and Delta Lake environments.

The role emphasizes data quality, production readiness, and collaboration with business and platform teams. A strong background in ETL/ELT design and scalable data solutions is required.

Qualifications

• 5+ years of hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• Experience with GCP, especially BigQuery.
• Knowledge of Delta Lake and data lake architectures.
• Strong ETL/ELT, data pipeline design and data quality.

Responsibilities

• Design, develop and optimize scalable ETL/ELT pipelines using Databricks and PySpark.
• Build agentic workflows to automate data engineering and operations.
• Integrate Databricks, GCP, BigQuery and Delta Lake environments.
• Develop reusable data frameworks and components.
• Implement data quality checks, monitoring and production controls.
• Optimize PySpark/SQL workloads for performance and reliability.
• Participate in testing, deployment, and production support.
• Collaborate with business and platform teams to identify further automation.

Skills

Databricks

PySpark

SQL

GCP BigQuery

Delta Lake

ETL/ELT

Data pipelines

Data quality

Troubleshooting

Software engineering practices

Tools

BigQuery

Delta Lake

Databricks

Job description

AgileEngine is seeking a Data Engineer to design and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL. You will build agentic workflows to automate data engineering tasks and integrate Databricks, GCP, BigQuery and Delta Lake environments.

The role emphasizes data quality, production readiness, and collaboration with business and platform teams. A strong background in ETL/ELT design and scalable data solutions is required.