Data Engineer: Databricks, PySpark & GCP Pipelines

AgileEngine, LLC.; Granada; Hybrid; EUR 107,000 - 143,000; Full time

Benefits offered by this job

Professional growth

Competitive USD-based compensation

Exciting projects with Fortune 500 and

Flextime

Job summary

AgileEngine, LLC. seeks an experienced Data Engineer with strong Databricks and GCP expertise to build scalable ETL/ELT pipelines and productionize agentic workflows for data automation.

You will design and optimize pipelines with Databricks, PySpark, and SQL, integrate with BigQuery and Delta Lake, and collaborate across teams to deliver robust data solutions for Fortune 500 brands and leading startups. Flexible work options available, with rapid growth and mentorship opportunities.

Qualifications

• 5+ years of hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills required.
• Hands-on GCP experience, especially BigQuery.
• Experience with Delta Lake and modern lakehouse architectures.
• Strong ETL/ELT design, data pipelines, performance optimization, and data quality.
• Proven ability to build reliable, scalable production data solutions.
• Solid analytical and troubleshooting skills; familiarity with testing, version control, deployment, and monitoring.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines to support new datasets and business needs.
• Create reusable data engineering frameworks for cross-use across projects.
• Implement data quality checks, monitoring, and production controls.
• Tune PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, and productionization of data solutions.
• Troubleshoot complex data issues and implement durable fixes.
• Collaborate with business, data engineering, and platform teams to identify automation opportunities.

Skills

Databricks

PySpark

SQL

BigQuery

Delta Lake

ETL/ELT

GCP

Tools

Airflow

Job description

AgileEngine, LLC. seeks an experienced Data Engineer with strong Databricks and GCP expertise to build scalable ETL/ELT pipelines and productionize agentic workflows for data automation.

You will design and optimize pipelines with Databricks, PySpark, and SQL, integrate with BigQuery and Delta Lake, and collaborate across teams to deliver robust data solutions for Fortune 500 brands and leading startups. Flexible work options available, with rapid growth and mentorship opportunities.