Data Engineer: Databricks & GCP Pipelines | Remote

AgileEngine, LLC.; Braga; Hybrid; EUR 70,000 - 100,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation (USD-based)

Exciting projects

Flextime

Job summary

AgileEngine LLC in Portugal (Braga) is seeking a Data Engineer with strong Databricks and GCP experience to design and productionize scalable ETL/ELT pipelines. You will develop agentic workflows and integrate with Delta Lake environments to support data automation.

Ideal candidates have 5+ years with Databricks, PySpark, and advanced SQL, plus hands-on BigQuery experience. The role offers flexible timing, remote/office options, and opportunities to work on Fortune 500 scale projects.

Qualifications

• 5+ years hands-on experience with Databricks and PySpark.
• Advanced SQL and data-processing skills.
• Hands-on experience with GCP, particularly BigQuery.
• Experience with Delta Lake and modern data lake/lakehouse architectures.
• Strong understanding of ETL/ELT, data pipeline design, performance optimization, and data quality.
• Experience building reliable, scalable, production-grade data solutions.
• Strong analytical and troubleshooting skills.
• Understanding of software engineering practices, including testing, version control, deployment, monitoring, and production support.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operational processes.
• Integrate agentic capabilities with existing Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines and processing solutions to support new business requirements and datasets.
• Build reusable frameworks and components across data engineering use cases.
• Implement data quality checks, monitoring, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, and productionization of data solutions.
• Troubleshoot complex data and production issues and implement sustainable solutions.
• Collaborate with teams to identify further automation opportunities.

Skills

Databricks

PySpark

Advanced SQL

GCP

Delta Lake

ETL/ELT

Data quality

Performance optimization

Production-grade data solutions

Tools

BigQuery

Job description

AgileEngine LLC in Portugal (Braga) is seeking a Data Engineer with strong Databricks and GCP experience to design and productionize scalable ETL/ELT pipelines. You will develop agentic workflows and integrate with Delta Lake environments to support data automation.

Ideal candidates have 5+ years with Databricks, PySpark, and advanced SQL, plus hands-on BigQuery experience. The role offers flexible timing, remote/office options, and opportunities to work on Fortune 500 scale projects.