Data Engineer: Databricks & BigQuery Pipelines (Remote)

AgileEngine, LLC.; Valencia; Hybrid; EUR 65,000 - 90,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Remote & office options

Job summary

AgileEngine in Valencia is seeking a Data Engineer who will design and optimize scalable ETL/ELT pipelines using Databricks and PySpark. You will productionize agentic workflows and integrate with GCP, BigQuery, and Delta Lake to support data automation.

Ideal candidates have 5+ years of hands-on Databricks experience, strong SQL, and a track record of building reliable data solutions. The role offers flexible times, remote/onsite options, and collaboration with Fortune 500 brands.

Qualifications

• 5+ years hands-on experience with Databricks and PySpark.
• Advanced SQL and data-processing skills.
• Hands-on GCP experience, esp. BigQuery.
• Experience with Delta Lake and lakehouse architectures.
• Strong ETL/ELT, data pipelines, performance optimization, and data quality.
• Proven ability to build scalable, production-grade data solutions.
• Strong analytical and troubleshooting skills.
• Understanding of software engineering practices (testing, VCS, deployment, monitoring).

Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines with Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering tasks.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake.
• Develop data pipelines to support new business requirements.
• Create reusable frameworks and components for multiple use cases.
• Implement data quality checks, monitoring, and production controls.
• Optimize PySpark and SQL workloads for performance and scalability.
• Support testing, deployment, and productionization of data solutions.
• Troubleshoot complex data and production issues, delivering sustainable solutions.
• Collaborate with business and platform teams to identify more automation opportunities.

Skills

Databricks

PySpark

Advanced SQL

GCP

BigQuery

Delta Lake

ETL/ELT

Data pipelines

Data quality

Testing & CI

Version control

Deployment

Monitoring

Production support

Troubleshooting

Job description

AgileEngine in Valencia is seeking a Data Engineer who will design and optimize scalable ETL/ELT pipelines using Databricks and PySpark. You will productionize agentic workflows and integrate with GCP, BigQuery, and Delta Lake to support data automation.

Ideal candidates have 5+ years of hands-on Databricks experience, strong SQL, and a track record of building reliable data solutions. The role offers flexible times, remote/onsite options, and collaboration with Fortune 500 brands.